NewsIR 2016 (Behind the scenes)

About a week ago I attended the European Conference on Information Retrieval (ECIR). The conference was great and I will write a blogpost about it soon. However, the main focus of this article is one specific workshop within that conference: the Recent Trend in News Information Retrieval (NewsIR). The reason why I want to talk about it is because I was the lead organiser and the event ended up being a success much bigger than we could have predicted. This blogpost will explain how the workshop idea was born and how the workshop was organised. We thought it is worth sharing this knowledge hoping that other people can get some insight out of it. A latter blogpost will focus on the content of the workshop itself.

Inception
We have to go back one year back in time, to ECIR 2015, in order to understand how it all started. After a quite intense night of “networking” on the main bars of Vienna I realised that many researchers in the Information Retrieval (IR) community were using news articles as their input data to solve a completely different tasks, ranging from news recommendation to temporal summarisation or real-time clustering of news. However, our perspective and the collections that we use are very outdated and they usually represent a very biased version of the news. This could be in the form of not enough articles, only one or a few sources, or extremely cleaned and filtered content rather than the real noisy data you would consume in reality. This creates an unrealistic environment for evaluating IR models/tasks that does not exist in reality. In addition, the way news-related content has changed dramatically in the last decade and we have not have a focus workshop on the topic for years. Given this thought, the obvious next step was to chat with some of the researchers in ECIR 2015 and see if they will help or attend a workshop on the topic. Also, it was clear to me that I needed very experienced researchers on board to be able to create the workshop. I then used most of the conference breaks during the last day of the conference to confirm that there was a group of around 10 people who would be interested in the topic. Moreover, at that point, three very well-known IR researchers agreed to join me as co-organisers: Udo Kruschwitz, Gabriella Kazai and Frank Hopfgartner. Given that this happened in just about 2 hours, I was quite sure that we were into something. Nonetheless, we decided that the next step was to create a google discussion group for this matter and see if the community really wanted the workshop.

Growing and Preparation
The NewsIR Google group was created and within two weeks around 30 people have signed up and some of them were already interacting in the forum. It was then when we suggested that Signal would provide a dataset which represents the real landscape of news in a much more realistic way than previous collections. At this point, it was also clear that the community was very interested on re-visiting the news-related arena with a completely new pair of eyes and that we were craving for a new collection. Thanks to the forum, the community gave us so many new ideas and basically helped us shaping the call for papers and the focus of the workshop. Its main goal was to be debate environment to understand how to move forward and to prioritise and understand what are the main tasks we are solving and the main challenges we are facing. At this point, the group was about 50 people with representatives from the main universities and companies working in IR and I was humbled to see such interaction and help from everyone. In addition to this, by now the last co-organisers of the workshop were also added: Dyaa Albakour, David Corney and Ricardo Campos. Finally, based on all the suggestions from the forum, and with the strength of a clear interest behind it,  we submit the NewsIR as a workshop for ECIR 2016 and it was accepted with great feedback from the reviewers.

At this point, we had the usual challenges that every other workshop has. How to make a popular Call for Papers, how to organise the proceedings, … Luckily, we had a lot of people in the organising committee with an immense amount of experience organising previous workshops. A special mention has to go to Dyaa, who worked tirelessly in the weeks and months closer to the event to make sure that everything was perfectly organised. Also, we manage to have two amazing keynote speakers, Jochen Leidner and Julio Gonzalo. As a result, the workshop was very popular and we received a large number of submissions.

The next challenge we encounter was during the workshop itself when we had to coordinate a workshop with almost 70 registered attendees, 2 keynote speakers, a panel, dataset group discussions, 9 presentations and a poster session. The schedule focused on maximising the interaction between the participants. For this reason, we limit each paper to be presented in 10 minutes, while also being presented as a poster. This format was very well received and people were still debating in front of the different posters even after the (amazing) food was already served for lunch. If this is not a proof of the format and the quality of the papers I do not know what is…

Lessons learnt
The first and most important lesson we learnt is that it is much easier to involve the community early on when you are planning to organise a workshop. In our specific case, the people involved in the NewsIR google discussion group helped shaped the objectives of the workshop. Also, it makes easier compiling a Program Committee as several of the PC members for NewsIR were indeed members of the group. Secondly, if the group is popular, it provides strong evidence that the workshop will probably be popular also. This will positively affect the chances of acceptance for any conference, especially if some influencing members of the community are active part of the discussions. For the schedule of the workshop itself, we found that allowing 10 minutes per paper and additionally having a poster session is a very good combination for workshops.

I have learnt a lot throughout this process and I really hope that this blogpost can provide some help and insight. My last comment has to be a huge thank you to all the people  who were involved in the creation of the workshop in all the different stages.