The main goal of a medical congress is to make a real impact on the practice of its participants, by proposing a scientific programme that effectively contributes to improving patients’ health, both on a short- and long-term basis. However, the transfer of knowledge does not necessarily lead to a change of practice. So, how does one assess whether a congress has any real impact on its participants? And if it hasn’t, how can its educational value be enhanced so it achieves its goal?

“Top-down” plenary sessions no longer suffice

Studies in education science, especially those related to adult learning, are clear: it is not enough anymore to impart scientifically proven state-of-the-art knowledge. Top-down plenary sessions do of course have certain assets, but they should be complemented with other means to satisfy the specific needs and the learning capacity of adults. There are three principles to andragogy (which makes the learner an actor in their own educational experience): sessions should be problem-based, contextualised, and involve cooperative learning.

« The new media environment and evidence-based adult learning theory show that a new educational paradigm is required - one that is built around the active involvement of participants who become actors of their learning »⁽¹⁾

Adult learners can reap more benefits by responsibly placing themselves at the very centre of their learning experience.

Furthermore, according to the size and scope of the congress, other factors should be taken into consideration. We know for example that the nationality, cultural background and resources of participants can have a major effect on how they learn. Equally, physicians who have not been on an educational course for a long time can be less inclined to actively participate and share their experience, because they feel their practice is not state-of-the-art.

How can I measure the impact my congress has on the practice of its participants?

Several evaluation models exist but the one that is most frequently used in the medical field is the 4 Level Kirkpatrick⁽²⁾ model, that dates from the late fifties:

1. Reaction - The degree to which participants find the training favourable, engaging and relevant to their jobs
2. Learning - The degree to which participants acquire the intended knowledge, skills, attitude, confidence and commitment based on their participation in the training
3. Behaviour - The degree to which participants apply what they learned during training when they are back on the job
4. Results - The degree to which targeted outcomes occur as a result of the training i.e. When applied to the medical field: how does knowledge acquired at a congress affect the way how I treat my patients?

 

Evaluating impact – an ongoing challenge!

It is tricky to apply this entire scientific model to a field that is in constant evolution. Researchers are currently studying each of the levels and considering how to progress from one to the next.

Without realising it, most congresses already evaluate the first level, via satisfaction surveys that are carried out immediately after sessions take place.

The evaluation of the second level is a lot less common, but exists for example when sessions are part of a continued education course. In France the DPC - ‘continued professional development’ method entails the evaluation of knowledge before and after the curriculum. There are two objectives to this: to assess whether there was indeed an increase in competency, and to encourage the learner to clearly differentiate what they already knew and what they have learnt.

The third level is rather more complex: how to determine whether the participant has put their new knowledge into practice?

From knowledge to practice: six years of research led by Europa and a laboratory at the Jean-Jaurès University 

In 2013 Europa co-founded RiMEC (Réinventer le Média Congrès): a research laboratory that was awarded a grant by the French National Research Agency (CNRS) and accorded a business contract by the French region of Occitanie. It is based at the Jean-Jaurès University in Toulouse, France and is co-directed by Jérôme Sicchi (Development Manager of Europa Group) and Monique Martinez-Thomas (University Professor).

Its aim was to link the results of academic research to the reality of congresses, in order to find a way to evaluate participation in the simplest way possible. 

After several years of research and fieldwork⁽³⁾, the team elaborated a questionnaire that assesses whether this third level is attained: does the participant really put into practice what he/she has learnt?

The criteria include: Is the healthcare professional in a position to apply the new practice? Does he/she have the necessary equipment? Will their working environment be favourable to the change? Does the physician consider the knowledge transmitted during the congress to be of authoritative? Is the reputation of the congress such that the knowledge imparted can influence the healthcare professional? Does the participant consider the new practice to be ethical? Does he/she feel capable of change? etc.

Another line of research was to determine the best educational means to influence physicians and change their practice, by evaluating the organisation of the congress. A new name: « congréturgie⁽⁴⁾ » defines the need for change, with a particular focus on space, people and rhythm.  Tested on site, these innovations have been implemented in Europa congresses for many years now, and are radically transforming the transmission of knowledge.

The laboratory’s actions continue, notably with the organisation of a “congress about congresses” in 2020, to carry on inventing those of the future: www.qcvn.fr

What can be done to increase the educational quality of a congress?

Above all, one factor is essential: the need to encourage participants to share their practice, so it can be confronted with the proposed alternative. Everything must be done to stimulate interaction between the participants and the facilitator, and to establish a trusting relationship so the participant does not feel judged. 

Here are a few of the methods we implement at the congresses we organise:

• Ensure the target audience has been correctly identified for each session, and that the educational content and level is appropriate;
• Verify the usability of the educational content, so that participants can easily review the key messages when the session is over;
• Help participants to exchange their point of view in every possible way;
• Set up especially effective ‘demonstration’ sessions such as ‘serious games’, ‘interactive patients’, etc;
• Teach facilitators how to get the audience involved, how to be particularly attentive to participants’ needs, and play out the session accordingly. At Europa, our ‘training the trainer’ courses are called ViTAL. We coach the facilitators until they themselves are also able to train others;
• Change the session room set-up to reduce as much space as possible between the facilitators and the participants: the rooms are used horizontally, with a maximum of 6 rows of chairs. To promote a convivial atmosphere, the stage is also removed. The facilitator moves around the room, encouraging the participants to join in discussions;
• Enable facilitators to use digital tools for the transmission of messages and questions. With these tools, the audience can influence in real-time how the session proceeds.

Conclusion: towards a new educational standard?

The educational quality of a scientific programme and its capacity to transform practice - is crucial. Our experience has also confirmed several times that it works.

At Europa, we believe that the medical congress of the future will evolve as participants will want to be become more involved in their own education, and that governments and payers will become more demanding regarding the measurement of the actual value perceived from attending congresses. The level of satisfaction and the intention to change practice will therefore be systematically and statistically evaluated. This could lead to a standardised ranking of congresses, a sort of educational event impact factor, which participants and sponsors could use to help them decide which one to attend.  This is why we firmly believe in the value of measuring impact, evaluating educational value and transforming the congress experience - get ready for the future! 

 

(1). Marco, J., Spencer, A. & Breheret, M. Critical reflection on postgraduate learning: education through sharing. EuroIntervention J. Eur. Collab. Work. Group Interv. Cardiol. Eur. Soc. Cardiol. 13, 625–630 (2017).
(2). Kirkpatrick, D. L. (1959). Techniques for Evaluation Training Programs. Journal of the American Society of Training Directors, 13, 21-26.
(3). LESTRA Martin, MARCO Jean, PÉRAN Bruno, MARTINEZ Monique, AÏT Ali Cédric : « Let’s foward with evidence-based adult learning. An experiment with a group of early career international cardiologists at EuroPCR 2016 », in Eurointerventions, n°145, décembre 2018, p.1262-1268. DOMPEIX, Clémentine, BREHERET, Magali, SPENCER, Allan et MARCO, Jean : "The concept of participatory learning group development", EuroIntervention Journal, 2011, vol.7, n°1, pp. 29-31.
(4). JACINTO, Gilles, MAYER, Camille, DESCOMBE, Benoit, MARTINEZ, Monique et PÉRAN, Bruno : "La congréturgie : penser le congrès comme une scène. L'exemple d'EuroPCR 2016", in MARTINEZ, M. (dir.) : Le théâtre appliqué : enjeux épistémologiques et études de cas, 2017.