The 37 questions that ESOMAR has created are intended to give a standard set of questions a buyer can ask to determine whether a sample provider’s processes and samples align with their research aims.
Please ﬁnd our answers to the questions below.
Internet Research Bureau was established in May 2011 with the goal of collecting, researching, and disseminating information for B2B companies. The company has more than 12 years of experience serving market research agencies. its management team and senior executives are industry veterans, each with 15+ years in the MR industry. We do not provide direct marketing services. Our services include market research-related services only.
We have staff responsible for developing and monitoring the performance of the sampling algorithms requisite to the management of our B2B data collection projects. Their training in sampling techniques is appropriate for their expert delivery of B2B projects with detailed target and quota specifications. Such training is provided over an initial three-month period for each project manager and is thereafter ongoing, including regular updates on guidelines as and when required, to maintain best practice.
Methodologies include stratified and targeted sampling methodologies for optimal delivery per project. More detail on this can be provided if needed.
As far as possible, delivery and collection controls are system-controlled and automated, providing homogeneous sampling quality. Our QA team works with our software engineers to refine the tool to monitor and enhance the response automated sampling function and algorithms.
Our sampling management tool has been developed internally for over a decade and the software is fully owned and operated exclusively by IRB. It is known as PASMT (Project and Survey Management Tool) and is designed to reduce duplication and increase data quality.
In addition to online sampling, IRB also supports clients with survey programming, hosting, translation services, and data processing. IRB recruits for IDIs, FGDs, and CATI surveys. We also provide data analysis services on a case-by-case basis.
In addition, IRB subcontracts further research services where required including quality moderation and analysis.
Sample Sources and Recruitment
Internet Research Bureau owns and maintains an online survey panel website named
Internet Research Bureau owns and maintains an online survey panel website named ‘Opinion Bureau’. We source users from our panel, ‘Opinion Bureau’, and other panel companies as and when required. We recruit users for ‘Opinion Bureau’ from various sources such as search engines, affiliate marketing, referral programs, banner advertising, social media, etc. We have approximately 2.4M registered users in Opinion Bureau who participate in our surveys depending on their profile and the topic of the survey.
Opinion Bureau. We source users from our panel Opinion Bureau and other panel companies as and when required. We recruit users for Opinion Bureau from various sources such as search engines, affiliate marketing, referral programs, banner advertising, social media, etc. We have approximately 2.36 M registered users in the Opinion Bureau who participate in our surveys depending on their profile and topic of the survey.
Internet Research Bureau provides about 50% of the samples from our proprietary panel, Opinion Bureau – a panel community that we manage ourselves.
We recruit users for ‘Opinion Bureau’ from various sources such as search engines, affiliate marketing, referral programs, banner advertising, social media, etc. There is a good mix with most of them coming from affiliate marketing. Recruitment is open to everyone within the given market from 15+ or 16+ depending on the market. The usage of these channels is similar across all markets.
Panelists need to pass through our quality checks which include their digital fingerprint, geolocation, proxy, and behavior analysis. We also carry out manual checks for each panelist. Registered members are checked periodically based on their behavior in survey, in response to our internal surveys and QC feedback from the clients and if they are found to be fraudulent, their membership rights are revoked. We conduct quality check surveys once a month to ensure that we do not send out surveys to members who respond inappropriately.
Opinion Bureau members can access our surveys by three means – Email (40-50%), Portal (50-60%), and App (less than 1%)
We don’t provide direct access to the client as we provide a managed service. We control the sources at our end and if any sources need to be restricted, our client needs to inform us in advance. We do not provide API services to the client so the control is at our end and it’s managed manually on a case-by-case basis.
We undertake detailed profiling of our samples and keep it up to update. Based on the research criteria, we pull the sample and based on the research requirement, we indicate the length of the interview and a compatible device to the respondent. So, they are ready for the shorter/longer questionnaire and will participate from the respective device only. In terms of matching samples to types of research, device type, or length, as we own our sample and have comprehensive records of panelist survey history, we can pull appropriate samples. As the question indicates, this can be useful to maximize quality, for example for very long questionnaires or minimize dropout from e.g. focus groups.
Our panel is primarily suitable for: –
- B2B/ B2C/patient surveys.
- All devices: mobile, tablets and desktops.
- Video surveys
- IDI recruitment.
- Short to medium length of online surveys with limited open-end questions
However, our panel is also available for panel building, community recruitment, focus groups, and
other long-term engagement activities.
Sampling and Project Management
Panelists are selected by our survey management application (PASMT) based on their demographic information such as age, gender, income, education, employment status, location, and profile information as defined in the questionnaire/screener provided by our client. Once the panelists are shortlisted, they receive an invitation from our system. Surveys matching the panelist’s profile are also visible in their portal which they access through their account. Standard email templates are used in all markets in local languages.
We do on occasion recommend quota controls where these are lacking in the client’s targeting and quota instructions, for which they may be appropriate to the research requirement. For example, if the client is instructing us to simply ‘gen pop’ for a consumer project, or ‘executives’ for a B2B project we will discuss with them to ensure that the dataset will support the analysis required with demographic and/or firmographic targeting and/or quotas.
IRB does not source profile information from any third party. Once a panelist is recruited to our panel, we collect their demographics and other profiling information in different steps.
We collect the following information when members register to our panel: First Name, Last Name, Email ID, Date of Birth, Gender, Household Income, Highest Level of Education, Employment Status, State/Province, Physical Address, Zip Code/ Postcode, Phone Number (Optional).
We ask our members to keep their information updated and encourage them to do so by sending them to their profiling page after completing each survey. We also conduct continuous extensive profiling of our panel members: in addition to the information, respondents provide at the time of registration, other profile information is collected, after joining, based on the below categories: Personal and Family, Education, Occupation, Health and fitness, Finance, Leisure and Activity, Food and Beverage, Household appliances, Automotive, Travel, Media, Games.
We also have a system to use pre-screening in the surveys to continuously update profile information.
We hold basic demographic, firmographics, and health condition information on more than 80% of our panelists.
We can append answers for demographic or firmographic data points, generally supplying these free of charge for up to 2-3 data points.
We need the following information to provide an estimate of feasibility:
- TG Definition
- Sample description
- Project description
- Data collection methodology
- Any de-dupe criteria from past surveys
- PII collection if there is any
- Device compatibility
- Screeners/ Questionnaire
- Quota details
- Incidence rates
- Length of interview
- Field work timeline
- Sample distribution across fieldwork time
If we are unable to complete a project, we would take all necessary steps to ensure the client understands the field difficulties and knows the best way to complete the fieldwork. The steps include:
- Field work extension
- Quota relaxation
- Screener relaxations
- Price increase for better incentivization
- Inclusion of all possible third party/sub-contractors – We don’t necessarily inform the sample buyer about individual sub-contractors unless they have specified that we should do so.
Our survey router assigns the survey randomly to the users who land on our survey router page irrespective of allowed sources. The survey is assigned if their demographics and profile information match the screening criteria of the survey. A user can choose to route to a survey or decline to do so. User can route to as many surveys as they want based on the availability of the surveys. We can also control and limit routing on the survey level as well as the source level.
There is no limit on participation through the router but there are time limits on qualifying for a survey. The qualification limit varies by country and source.
We give the following information to our proprietary panel participants: survey, survey category/topic, length of the interview, reward amount, survey instructions, survey link, unsubscribe link, help link, terms and condition link, privacy link, a welcome message with the declaration of voluntary participation, honesty, and integrity. If the user is coming from a third-party source, we show them the help link, terms and conditions link, privacy link, and welcome message with the declaration of voluntary participation, honesty, and integrity.
Yes, we do allow users to choose from a list of available surveys if they log in to the portal or app. They choose surveys based on the topic, reward amount, and length of the interview. They can see this information and decide which survey to take.
We start with the standard incentive and can increase or decrease the incentive at any point that it’s needed. However, we ideally don’t change the incentive frequently (max up to 3 times) during the field. An update to the incentive is updated on the panelist level so it can be flagged who qualified at which incentive amount.
No, we don’t measure participant satisfaction at the project level. But we provide continuous helpdesk support for our members and encourage panelists to send their feedback and queries, which helps us to improve the user experience and satisfaction levels of our members. We closely monitor their feedback and answer their queries to maintain high satisfaction levels. Some of our clients do measure parameters like the topic of the survey, length of the survey, ease of design, user experience and questionnaire understanding, relevancy of questions, etc.
No, we don’t provide any report as such until asked by the client. Sometimes clients do ask for the debrief report such as invitation sent, channel report, response rate, qualified, terminates, dropouts, quota-full, field report, conversion rate, incidence rates, rejection rate, the median length of the interview, etc. We can provide this information on request.
Data Quality and Validation
A participant can’t participate in the same survey twice irrespective of their source. Once the participant has taken a survey, he can’t enter the same survey again as their member ID is updated in the survey. If the user is coming from third-party sources we track their device id, sub-ID, and cookies to ensure they don’t enter the same survey again.
On the other hand, we keep track of the survey behavior of all our members and as a quality control measure we don’t let a user participate in more than 30 surveys per month (one survey invitation a day). Also, when monitoring the high qualifying rates of any user, we scrutinize their responses in the survey and eliminate them if they are flagged as a multiple survey taker.
Yes, we maintain all the information about recent participation history, date of entry, source, screener response if applied, IP-related information, device information, etc. of the survey participants and this can be shared with the client if requested.
The panelist’s identity in a survey is verified by their panelist ID, a unique ID that is assigned to panelists when they sign up with our panel, which enables us to track them across all their survey activities. When the survey invitation is sent to a panelist, the survey link remains unique for the participants, containing the panelist ID in encrypted form. If anyone tries to manipulate the link or change the panelist they wouldn’t be able to enter the survey as their link would become invalid and stop them from entering the survey. Their account is also locked by their email ID and password so if they wish to participate in a survey via the portal or mobile app, they would be required to log in using their email address and password. They can’t enter the survey portal until they log in securely and successfully.
We mirror the sampling methodology across wave trackers to ensure a decent sample blend and source consistency. Our sampling tool has a feature to set the limit for each source at the survey level as well as the quota level. So, we can copy the sample distribution by source in case of tracking studies and we can also set a good sample mix for ad-hoc surveys to ensure a decent blend. We keep track of sources and can be appended if requested.
We have developed a quality score on a scale of 1 to 5 for each panelist who joins our panel. If their quality score drops below 3.5, they are restricted to participate in B2B surveys and if it drops below 2.5, they are taken out of the survey pool and blocked with a notice to the panel member. The score is derived from a few parameters such as security terminates, client rejections, profile completion, screener response, etc.
Also, if a panelist’s response is rejected in 3 consecutive surveys by our clients, they are immediately removed from the panel.
We check and clean the data before sharing the raw data with the client. We have set a process to check every data individually and flag them on the following parameters:
- Open-end questions
- The illogical or irrelevant response pattern
- Missing data
- Contradictory responses within surveys
Our project managers flag respondents on the above parameters and remove and replace those responses with new data before sharing it with the client. Any questionable or suspicious response is removed manually before sharing.
Policies and Compliance
- Compliance to protect the privacy of survey participants in accordance with International and local regulations including but not limited to EU-U.S. and Swiss-U.S. Privacy Shield Frameworks,
- GDPR, and ISO.
- Types of PII collected along with its usage.
- How data is processed, stored, retained and shared
- Cross-border data transfer policy
- Rights of panelists, legal bases
- Access and control of PII by panel members
We have designed our data protection compliance program to meet the data protection and privacy laws and regulations. This is in accordance with international and local regulations including but not limited to EU-U.S. and Swiss-U.S. Privacy Shield Frameworks, GDPR, and ISO standards of privacy.
On the technology front, we keep our panel member’s data on a very secure server, which can only be accessed by authorized team members. We do not share any personally identifiable information relating to our panel members with our clients or any other third party without the panelists’ consent. Our websites and server are also secured by the SSL certificate of DigiCert.
Our primary owned source of survey participants is opinionbureau.com. We don’t provide any
Children, as defined in law by each territory, are not eligible to join our panel or participate in our surveys. But, if we must contact minors defined by local or country law, we seek parental consent before letting them enter the survey.
We comply with national/regional regulations including GDPR, COPPA, ESOMAR, MRS Guidelines, and national and international MR association guidelines.
IRB implements data protection by design by –
- Outlining technical and security measures
- Organizational processes and protocol for data execution
- Defining data handling and data sharing practices
- Defining Dos and Don’ts for the employees and people managing data in any form.
- Defining and limiting access to relevant people
Internal approval system for access to data
IT system, services, and contacts
Organizational and employment policies
No, however, we are applying for this in 2023.
- Average qualifying or completion rate, trended by month
- Percent of paid completes rejected per month/project, trended by month
- Percent of members/accounts removed/quarantined, trended by month
- Percent of paid completes from 0-3 months tenure, trended by month
- Percent of paid completes from smartphones, trended by month
- Percent of paid completes from owned/branded member relationships versus intercept participants, trended by month
- Average number of dispositions (survey attempts, screenouts, and completes) per member, trended by month (potentially by cohort)
- Average number of paid completes per member, trended by month (potentially by cohort)
- Active unique participants in the last 30 days
- Active unique 18-24 male participants in the last 30 days
- Maximum feasibility in a specific country with nat rep quotas, seven days in field, 100% incidence, 10-minute interview
- Percent of quotas that reached full quota at time of delivery, trended by month
We have metrics available for a variety of measures, including completions, conversion rates, and device usage to name a few. We can provide the required metrics upon request since these are dynamic in nature.