Study design

We conducted a cross-sectional, web-based, open survey aimed at patients (health service users), patient representatives, scientists, healthcare professionals and representatives of the private sector, that are involved in studies of interventions that are already widely available. We reported this study in accordance with the Checklist for Reporting Results of Internet E-Survey (CHERRIES) [13]. Ethical approval was obtained from the Ethics Committee of the Albert-Ludwigs-University Freiburg (No.294/20).

Questionnaire

In the course of the systematic literature searches we did not identify any suitable, validated questionnaire for the specific purpose of our study. Therefore, we designed a de-novo self-administrated questionnaire in German language. Answer options were developed on the basis of a literature research and three expert consultations. Questions with single- or multiple-choice and free-text fields were used. Willingness to participate in RCTs and OSs, respectively, was measured with visual analogue scales ranging from 0 to 100 with higher values indicating higher willingness.

The 51-item questionnaire was sub-divided into five sections: (i) identification of participants as stakeholder, (ii) general willingness to participate, support or carry out studies, (iii) willingness to participate, support or carry out RCTs or observational studies, respectively, (iv) demographic and professional background including experience in years (for patient representatives and representatives of the private sector) or number of engagements in studies of medical research (for patients, healthcare providers and scientists), and (v) additional comments. With the help of a survey-logic, the identification as a specific stakeholder determined which items of the questionnaire were displayed in the subsequent pages. Items, vocabulary and expressions were selected carefully in order to prevent linguistic miscomprehension which could lead to false responses. In multiple-choice questions, answer options were displayed in a random order to reduce bias due to the primacy effect [14]. The selection was restricted to three answer options in order to identify the most relevant motives. The questionnaire with all items is attached in the S1 Appendix.

We did not perform psychometric testing to validate our questionnaire. However, we carried out a pre-test of the questionnaire with 12 members of our working group and experts in health research methods. The pre-test aimed to identify potential issues in the functioning and wording of the survey, and to finalise all answer options.

Survey administration

The questionnaire was created on the online platform SoSci Survey [15] and it was open for participation for five weeks, from 15th June to 19th July 2020. A reminder was sent two and four weeks after initiation of the survey. Participation in the study was voluntary, and respondents could terminate their participation in the survey at any time and without giving reasons. The survey ended when the respondent closed their browser window. No reward was offered for participation, but the opportunity to be informed about the results of the study was provided. No strategy to avoid multiple entries was implemented in order to guarantee the anonymity of the participants. Study information, information about data protection, and the contact information of the investigator were presented in the survey introduction. No personal but only anonymised data were collected and analysed for the purpose of this study. Therefore, no informed consent for the study was necessary according to the standards of the Ethics Committee of the Albert-Ludwigs-University Freiburg and the data protection officer. Respondents automatically participated by progressing to the first page of the survey.

None of the questions of the survey were mandatory, and all except for the question used to identify the participants’ stakeholder group(s) could be skipped. In the case of non-selection of any answer option, a note appeared to encourage the participants to answer. A button to return to the previous page was displayed to allow respondents to review and change previous answers.

Sampling

Members of the following stakeholder groups were the target population of the survey: (i) patients or patient representatives, (ii) healthcare providers (physicians and health professionals), (iii) scientists and trialists in medical and health science, and (iv) representatives of the medical or pharmaceutical private sector (e.g. medical technology or the pharmaceutical industry). No geographical limitation was applied. However, as the questionnaire was written in German, knowledge of the German language was necessary to participate in the survey.

We used a snowball system for recruitment. Representatives of German associations and societies of the stakeholder groups were invited to take part in the survey and to forward the invitation to members of their network. A google search was performed to identify relevant associations and societies representing the target stakeholder groups in Germany. In order to recruit patients and patient representatives, the office of patient representatives in the German Federal Joint Committee (Gemeinsamer Bundesausschuss, G-BA), as well as the four organisations that are currently entitled to appoint patient representatives to the G-BA were contacted. Healthcare providers (physicians, dentists, psychologists, nurses, allied health professionals) were contacted via associations representing the interests of their respective profession. Scientists and trialists were recruited via representatives of the German network for evidence-based medicine, the German network for healthcare research and the Clinical Trials Unit of the University of Freiburg. Trade organisation for the medical technology and pharmaceutical industries were approached in order to invite representatives of the private sector. The contacted associations and societies are listed in detail in the S2 Appendix.

We disseminated email invitations to the target organisations via public contact details. The invitation included a brief introduction to the study, an approximate completion time and the link to the survey. In addition, an invitation to the survey was disseminated via Facebook and Twitter accounts of Cochrane Germany and their newsletter [16].

Data management

Data were collected, documented and structured on the basis of an a priori developed study protocol. Data were managed by the research team. All data were collected anonymously. The collected data were coded for statistical analysis and will be stored for five years in the password protected server environment of the Albert-Ludwigs University, Freiburg.

Data analysis

Data were downloaded from SoSci Survey once the survey was closed. Participants who provided at least two valid responses were included in the analysis. However, respondents who did not identify themselves as belonging to one of the stakeholder groups and who did not answer at least one additional question were excluded from the analysis. All valid responses were tabulated, and statistical analyses were conducted using Microsoft Excel 2010. Analyses were performed for each stakeholder group individually. The participation rate was evaluated by dividing the number of people who had completed at least one survey page by the total number of visitors to the survey. The completion rate was calculated using the ratio of people who had submitted the final questionnaire page and those who had completed the first question page.

Motives were described according to their frequency and percentage, and were ranked accordingly. Answers to the visual rating scales were divided into five groups and recoded as follows: “1 to 20” as very low, “21 to 40” as low, “41 to 60” as moderate, “61 to 80” as high, and “81 to 100” as very high. For comparison between two groups an independent t-test was used for a response frequency n > 30, and the Mann-Whitney-U-Test for n < 30. Results with p < 0.05 were considered to be statistically significant. Qualitative data from the free-text responses were analysed inductively by coding the data and creating themes.

Demographic characteristics of the sample

Data of 46 patients and patient representatives, 72 healthcare providers, 48 scientists and 13 representatives of the private sector were included in the analysis; 24 of the participants identified themselves as belonging to two or three stakeholder groups. The characteristics of each stakeholder group are presented in Tables 1–3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Characteristics of the sample.

https://doi.org/10.1371/journal.pone.0271791.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Engagement of the respondents in medical research (in number of studies)^a.

https://doi.org/10.1371/journal.pone.0271791.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Experience of the respondents in medical research (in years)^a.

https://doi.org/10.1371/journal.pone.0271791.t003

The majority of respondents were female (52.6%), aged between 35 and 54 years (46.8%), and reported to have little experience with regard to participation or engagement in medical research.

General willingness to engage

The majority of the respondents reported to be willing, in principle, to engage in studies of benefit assessment of interventions that were already used in practice. Forty-two patients (91.3%) were willing to participate; all patient representatives (n = 17) and 68 of the healthcare providers (95.8%) were willing to support, and 40 of the scientists (87.9%) were willing to carry out studies for benefit assessment post-entry. Ten out of thirteen respondents of the private sector (76.9%) were also willing to support, but only four of them were willing to carry out such studies.

When asked for their motives for engagement, personal interest and altruistic motives such as the improvement of healthcare or interventions dominated. Burden associated with the engagement and lack of resources and finance were commonly mentioned as reasons against engagement. However, eleven (24.4%) patients, eight (47.1%) patient representatives, eleven (15.5%) healthcare providers and four (9.8%) scientists indicated that they did not see any reasons against participating, supporting or carryout. The five most frequently mentioned motives in each stakeholder group are displayed in Table 4. All motives with their frequency of being mentioned are listed in S3 Appendix.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Principal motives for or against engaging in post-entry studies (Top 5).

https://doi.org/10.1371/journal.pone.0271791.t004

Willingness with regard to study type

Fig 1 shows the self-reported willingness to engage in post-entry studies in each stakeholder group in RCTs and OSs, respectively. Except for the private sector, the majority (at least 60%) of the respondents reported high or very high willingness to participate, support or carry out both study types. The mean score for willingness to participate in RCTs was 65.5 points (SD 28.5) as compared to 75.7 points (SD 25.1) in OSs. The difference in willingness to participate in RCTs or OSs was not significant (p = 0.08). Regarding willingness to support RCTs by patient representatives the mean was 78.7 points (SD 26.3) With regard to supporting OSs, the mean was 73.2 points (SD 26.5). No significant difference in willingness to support RCTs or OSs was revealed (p = 0.08). Regarding willingness to support by healthcare providers, the mean was 78.7 points (SD 26.3) and 73.2 points (SD 26.5) for RCTs and OSs, respectively. The difference in the willingness to support between RCTs and OSs was not significant (p = 0.24). The mean score for willingness in respondents of the private sector to support was 42.0 points (SD 26.9) for RCTs and 60.8 points (SD 30.0) for OSs. The difference was not significant (p = 0.23). The mean of willingness of respondent from the private sector to carry out a study was 26.6 points (SD 34.8) for RCTs and 33.3 points (SD 30.8) for OSs. The difference was not significant (p = 0.42). The mean of willingness of scientists to carry out a study was 85.3 points (SD 22.8) and 71.0 points (SD 28.6) for RCTs and OSs, respectively. The difference between RCTs and OSs was statistically significant (p = 0.01).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Willingness with regard to study type.

Red: randomised controlled trials; blue: observational studies.

https://doi.org/10.1371/journal.pone.0271791.g001

The most frequently selected motives which referred to specific study types are displayed in Tables 5 and 6. All motives with their frequency of being mentioned are listed in S4 Appendix.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Motives against engagement in randomised controlled trials.

https://doi.org/10.1371/journal.pone.0271791.t005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Motives against engaging in observational studies.

https://doi.org/10.1371/journal.pone.0271791.t006

Summary of the key results

Data of 46 patients, 72 healthcare providers, 48 scientists and 13 representatives of the private sector were analysed. In general, the majority of patients, patient representatives, healthcare providers and scientists were willing to participate or engage in studies, also with regard to a specific study type (RCTs and OSs). The most important motives for or against participation or engagement were similar across the four stakeholder groups. Personal benefit and interest, as well as benefits for others including the desire to contribute to the improvement of interventions and healthcare in general were most frequently mentioned. In contrast, poor integration into daily life of tasks associated with the studies, expectations of strain and negative impacts, and lack of trust in study personnel were mentioned as reasons against participation or engagement. At least 60% of all individual stakeholder groups showed high to very high levels of acceptance for both RCTs and OSs.

Representatives of the private sector varied in their willingness to support RCTs and OSs. For both study types, they were motivated by a wish to progress their own methods, to facilitate marketing, and to contribute to the improvement of the quality of methods and healthcare. However, they were concerned about additional costs and source of finance, and considered that there was no necessity for the studies since there had already been sufficient evaluation pre-entry.

Interpretation

Based on the high self-reported willingness to participate in, support or carry out post-entry studies, positive engagement can be assumed on the part of patients, patient representatives, healthcare providers and scientists when planning and conducting post-entry studies for benefit assessment. This finding is conflicting with findings in the scientific literature that participant recruitment and adherence to studies is challenging in many instances, and a substantial proportion of studies fail due to difficulties in recruitment [17, 18]. The result of this study could indicate that the recruitment of patients or healthcare providers might be easier if the intervention is already available. Due to wide availability, potential participants might consider interventions as safe, and therefore may be more inclined to participate or promote participation in studies [19]. However, it should be noted that only general questions about willingness were asked and no specific scenario was presented. Thus, inquiring about the willingness of hypothetical studies might have caused respondents to report higher willingness. Furthermore, respondents in the group of healthcare providers lack individual experience in medical research and might not be aware of these challenges in practice.

Knowledge about stakeholders’ motives for and against participation and engagement is important in practice since it can serve as starting point for overcoming challenges or barriers in planning and implementing post-entry studies. Personal benefit and altruistic motives were key themes in all stakeholder groups that are also present in current literature [11, 20]. Information on the relevance, necessity and potential benefits of a specific study might positively affect potential volunteers’ decision to participate, and potential partners’ (healthcare providers, patient representatives) decision to engage. Therefore, in the run-up to any study, emphasis should be put on the provision of information and education of stakeholders. This strategy is especially important in post-entry studies, since, as in this survey, health service users or providers might not see direct benefits for themselves or their patients, respectively, or benefits for future health service users since interventions are already wide available.

With regard to barriers to participation or engagement, many participants identifying themselves as patients or healthcare providers mentioned poor integration into daily life of study tasks and high levels of inconvenience as common reasons for concerns. This is in accordance with findings in existing scientific literature that point to additional visits or distances to trial or examination centres as relevant factors for participation [12, 19, 21]. In order to facilitate the implementation of studies, researchers might need to design easily feasible ways to integrate study tasks into daily life—for health service users, but also for recruiting or examining parties.

Stakeholders’ perceptions of study type characteristics such as lack of control of intervention allocation due to randomisation is an important source of uncertainty regarding participation in RCTs, as it is also highlighted in a recent Cochrane review [19]. This may be due to a lack of understanding of the concept of randomisation and the principle of clinical equipoise between different study arms. When potential study participants assume clear differences in health outcomes between intervention and control group, for example, respondents’ concerns of not receiving the best intervention available and having poorer outcomes in the control group are comprehensible. Furthermore, treatment preferences are a key factor in the decision to participate in trials [19], but these preferences cannot be taken into account when group assignment is random. However, since we did not provide details of a specific study in this survey, it remains unclear which type of interventions and control interventions the respondents pictured when answering the question, and to which comparison the concerns are linked. Future research that specifies details of the comparator (active or inactive) might therefore be indicated. This could contribute to resolving the paradox between high willingness of study participation as reported in our survey and other research findings that demonstrate a reluctance of study participation in general [19].

Only few representatives of the private sector took the survey. It remains unclear whether the invitation was not forwarded by intermediaries of pharmaceutical organisations/ associations, for example due to a lack of relevance or permission, or whether the topic was generally seen as not relevant by this stakeholder group. However, willingness seemed to be linked to the aspects of financing; and respondents appeared to be unwilling to conduct a study if funding had to be provided entirely by their company.

Strengths and limitations

Strengths of the survey include our rigorous method of implementation. The survey was conducted online, which allowed participants to respond at their own pace. Honest responses were encouraged by the online-based conduct of the survey. Each participant was able to conduct the survey in a private setting, removing themselves from the possible influence or judgment of third parties regarding the responses. Another strength of the study was an appropriate number of participants from three of four stakeholder groups.

This study also presents several limitations. First, we used a self-administered questionnaire which has not been evaluated or validated as a measurement instrument. Discrepancies in how the principal concepts in the survey were presented and understood, respectively, may have led to inaccurate responses. Although the main concepts of the survey were explained in short texts within the questionnaire and the survey was piloted and tested among several participants, the possibility of misunderstandings cannot be excluded.

Second, the recruitment process created a selective study sample. Any generalisation of results must, therefore, be applied with caution. Drawing a convenience sample may have resulted in more people feeling attracted to the survey by the invitation to participate who were interested in the topic. Individuals who are not interested in the topic and their perspectives may accordingly be underrepresented. Older respondents and those who are low users of the internet, as well as those not standing close to medical associations and societies, might be underrepresented. We do not know how many of the intermediaries, associations or societies that were contacted had passed on the invitation email to members of their network although reminders were sent.

Third, in order to guarantee the anonymity of the participants, no strategy to avoid multiple entries was implemented. However, this may have created bias in the results since it was possible for people to complete the questionnaire multiple times to emphasise their views.

Fourth, respondents were not surveyed about real life situations, but rather rated their general willingness. No specific scenario was presented. Also, no further background information on the baseline situation was provided that would have allowed for an assessment of the necessity (“clinical equipoise”) or relevance of a study by the participants. This may have resulted in the weighing of risks and benefits being too abstract, polarised, or not occurring at all, with participants accordingly indicating a high level of willingness. This error raises the possibility of response bias, since it is uncertain whether stakeholders would react similarly if they are asked to engage in a real study.

Fifth, the survey took place during the Covid-19 pandemic. Stakeholders may have been too preoccupied to participate in the survey, or may have considered the topic of the survey to be less relevant to them in the particular situation. Moreover, in a time of urgent need for medical interventions against Covid-19, participants may have been more in favour of benefit assessment post-entry in order to accelerate access to new interventions.