Comparing Qualtrics & Survalyzer

VU Amsterdam has a university-wide contract and processing agreement with Qualtrics. Any VU employee can sign into Qualtrics using their VU single sign-on details. FGB also has a faculty-wide contract and processing agreement with Survalyzer. Access to Survalyzer can be requested by contacting the Faculty Survey Administrator. For Survalyzer, there are a limited number of workspaces available to FGB, so the Survey Admin may have to reject your request for access.

Why are there two questionnaire tools?

Survalyzer was introduced originally because the faculty was able to set up a processing agreement with Survalyzer’s provider. At that time, the VU was unable to get a processing agreement in place with Qualtrics, which limited how Qualtrics could be used in the faculty, particularly for surveys collecting sensitive information. Both options now have processing agreements in place (covering the GDPR requirements), but each option still requires some enhancements to their security measures to allow for the highest risk data to be collected using only one of the options. In other words, if the content of your survey is high-risk (privacy category orange or red), it’s advised to use Survalyzer for collecting the directly identifying information (e.g. names, contact information, digital consent) and Qualtrics for collecting the other research data. This is explained further below in the section on secure use of these tools.

If your data are not high risk (privacy category yellow, green or blue), you can use either Qualtrics or Survalyzer for your entire data collection, whichever better suits your purposes. Survalyzer, however, should only be used when Qualtrics does not meet your needs since Survalyzer has a limited number of workspaces available.

To help you pick an option, you can review the technical specifications of Qualtrics & Survalyzer below:

Qualtrics Survalyzer
Questionnaire design options • User friendly
• Wide variety of question options/types
• Very flexible
• Routing is intuitively programmed
• Questions and response codes can be exported as a Word doc
• User response data can be exported to Excel, SPSS, CSV, TDF or XML files
• Data exported as SPSS are automatically labelled and coded		 • User friendly
• Many question types but not as varied as Qualtrics
• Not as flexible as Qualtrics
• Routing is intuitively programmed
• Questions and response codes can be exported as a Word doc
• User response data can be exported to Excel or SPSS
• Data exported into Excel include separate tabs explaining the question variable labels and response codes, as well as the filters and routing used in the questionniare
• Data exported as SPSS are automatically labelled and coded
Distribution options • Several options for survey distribution • Fewer options for survey distribution
Data storage • Data are stored on EU servers
• Data are backed up and stored for 90 days; any data deletions are only permanent after 90 days have elapsed		 • Data are stored on EU servers
• Data are backed up for 90 days. When data are deleted they are flagged for 90 days and then deleted, and a back-up of this data is saved for a further 90 days, i.e. data deletions are only permanent after 180 days have elapsed

How do I use Qualtrics and/or Survalyzer securely?

While both Qualtrics and Survalyzer have processing agreements and contracts with our institution, that just covers the legal requirements. There are extra steps you should take to keep the data you collect with these tools secure, especially when collecting higher risk data (privacy category orange or red).

Qualtrics Survalyzer
Data Privacy • Generally avoid using the email distribution system (i.e. where you upload a list of participant e-mails to automatically send out unique survey links)1. You must avoid this when the data you’ll collect are privacy category orange or red. This manual explains an alternative approach.
• Under “Survey Options” ensure that:
    • “Prevent Indexing” is selected
    • “Secure Participants’ Files” is selected
    • “Anonymize Response” is selected
• In “Survey Flow”, add an “End of Survey” block, then go to “Customize” in this block and select “Do NOT record any personal information and remove panel association”		 • Generally avoid using the email distribution system (i.e. where you upload a list of participant e-mails to automatically send out unique survey links)1. You must avoid this when the data you’ll collect are privacy category orange or red. This manual explains an alternative approach.
Data Integrity • If you are conducting a longitudinal study with multiple follow-up questionnaires, do not use the default question names; give each question a unique name
• Test the survey flow before sending the questionnaire to respondents
• Before sending out the survey, check the exported data for issues, such as incorrect response coding. If you are randomizing your survey blocks/questions, make sure when exporting your data to select “Export viewing order data for randomized surveys”
• Download and save a copy of the questionnaire to serve as documentation of your response codes, question labels and survey routing
• It is generally recommended to export the response data in a non-proprietary format such as csv. You can also export an SPSS or Excel copy if that’s easier for you to work with, but a copy of the raw data in csv format is strongly recommended		 • If you are conducting a longitudinal study with multiple follow-up questionnaires, do not use the default question names; give each question a unique name
• Test the survey flow before sending the questionnaire to respondents
• Before sending out the survey, test the exported data for any issues, such as incorrect response coding or issues with how questions are labeled (particularly if your questionnaire has complex routing depending on the respondents’ answers)
• Download and save a copy of the questionnaire to serve as documentation of your response codes, question labels and survey routing

What do I do when I’m done my research or when I’m done collecting data with Qualtrics/Survalyzer?

If your data collection is finished, you should export all of the data you need from your survey tool and store the data on an appropriate storage location (or locations).

Neither Qualtrics nor Survalyzer should be used for data storage during your research! And they definitely shouldn’t be used as an archive for long-term storage of your data after your research is complete!!!

Once you’ve safely exported all of your data and stored it in a secure location, you must delete the data from your workspace. Lastly, if you are using Survalyzer and you no longer need your workspace, contact the Faculty Survey Administrator to let them know that your workspace can be deleted (to make space for someone else).




  1. If your data are lower risk and you opt to collect directly identifying data (such as names, contact information and consent) simultaneously with the other research data, you should export the directly identifying data as soon as possible and delete them from Qualtrics/Survalyzer↩︎