Category Archives: Management

How to attract and retain the best research talent

During the last 4 and a half years, my main responsibility in Signal has been to create a world-recognized research team and to make sure we are always at the cutting edge of research. This piece summarises some of the learning over these years on how to attract and retain the best talent in the Data Science and Machine Learning community.

Every business hopes that the application of Machine Learning and Artificial Intelligence will open new opportunities for them, but this has lead to a spike in demand for AI expertise which is now significantly greater than the available talent. This has caused the salary for these roles to sky-rocket, especially in the US. We are also seeing a “brain drain” from top academic institutions to tech corporations that can afford to pay six-figure salaries to attract the best minds. However, I believe that some companies misjudge how to attract the best talent in the space. Every person has different priorities and motivations, and while money is obviously important, it is commonly not the dominant factor for data scientists.

Certainly, some of the best talent ends up in the largest tech companies (e.g., Google, Tesla, Apple), receiving lucrative salaries, but I would argue that salary is often not the key motivator. Big tech companies tend to offer a workplace with huge amounts of data and infrastructure, while start-ups tend to offer freedom, flexibility, direct input on the company’s products and services and the possibility of disrupting a market. Technologists are attracted to workplaces that offer interesting problem-solving opportunities, where they are surrounded by people who they respect and can learn from. If the salary and overall benefits meet a certain threshold, you might be able to attract and retain talent even when competing with much higher salaries at larger corporations. The reasons for choosing an employer are varied but include the following factors:

  • Large and complex datasets
  • Prefered technologies or methods (e.g., Reinforcing Learning)
  • Groundbreaking products and services
  • Solving a problem of personal interest
  • Projects with talented coworkers.

For instance, at Signal, we have access to “the world’s news”: hundreds of millions of articles are available for us to view and to analyse how the world perceives different events, brands and topics. This is an amazing dataset for analysis that has attracted talented people into the company. Many are attracted by specific technologies or areas of research, either because they want to develop expertise or because they believe in its potential. Using Machine Learning has been a major factor in the talent acquisition for Signal, but we have also failed to hire some brilliant researchers as we were not using some specific technology in our stack at that time. The sector you are operating in is also an incredibly important factor. For instance, some may want to work at companies related to education or animal conservation, companies such as charities, NGOs or government. Lastly, the product or service itself is critical because we are, after all, creators and want to be proud of what we build. Attracting talent will always be easier if your product is amazing (or you have plans to make it so), in the eyes of data scientists.

From a personal point of view, I like to put myself in the shoes of each person working in our Data Science team and understand what “success” means for them on an individual basis. In general, if we have the freedom to work on interesting problems, with autonomy and great people around us to learn from, we will be happy. Nonetheless, we are all slightly different and have different personal goals. For instance, some individuals put more focus on learning (especially those early in their career), while others wish to be part of the academic community. For them, working in a company that encourages attending conferences and publishing papers is key. Focusing on these two cases, you will lose people in the former case if they do not believe they are learning enough, and you will lose people in the latter if they do not have the opportunity to work on and publish good quality research.

At Signal, we have been very active in the community since our inception. Not only have we been involved in many London meet-ups (e.g., Text Analytics and PyData) but we have also published several research papers, and conducted workshops as well as attended many academic conferences.

Our drive for innovation and research is demonstrated by our “Visiting Researcher” programme. We have always believed that the best research comes from universities and that the best way to compete with our peers is to attract talent from these institutions in the most effective way possible. We constantly try to identify the best and most promising people in specific areas of research. In many cases, this includes Ph.D. students struggling to decide what to do next in their careers (i.e., start-up, big company or academia). Signal provides them with 6-12 months work experience at a startup that does applied research to be used in a product by real, paying customers. At the same time, they benefit from the fact that the research they do is also publishable and shareable in some form. In return, the company gets the best-in-class researchers for our team to learn from, as well as IP and specific components built during their visit. We also have a very strong MSc program, mainly with UCL and the University of Essex, where we propose and supervise specific research MSc projects related to our area of expertise.

Both of these approaches produce not only IP, publications and new components for Signal but have already resulted in a “domino effect” as our alumni recommend us to colleagues in countries as far as Australia. I am incredibly proud of the fact that in the last four years, we have had 23 people in our research team in some capacity. The best way to grow your team is to treat current and past data scientists well, and to strive to be well-respected in the community. This also includes the people who, for whatever reason, decided to leave the company. No matter how good a company or manager is, people decide to move on to find new challenges and adventures. You have to ensure the leaving process is handled in the best possible way; a great recommendation from someone who left your business is like recruitment gold dust.

In summary, attracting and retaining talent in AI is not easy, and it takes time and effort. You must create an attractive opportunity that sparks the curiosity of each candidate in an environment where they can learn and flourish. In my opinion, one of the best ways to find this talent is to be a proactive and well-respected member of the community.

Note: A similar version of this post has also been published in the Signal company blog.


Challenges integrating research in a commercial pipeline: the S.I.M.P.L.E. approach

The integration of research ideas in commercial products is one of the most difficult challenges companies are facing in the predictive analytics and data science space. This post reflects my thoughts about this topic and an explanation of the S.I.M.P.L.E. approach that we use in Signal to incorporate new models or ideas into our pipeline.

Before going deeper into this topic, I want to say that I know for a fact that I do not have all the answers (not ever all the questions) to this challenge, and there is a lot to be learnt from some of the giants of our sector such as Google, Netflix, Twitter or Amazon (to name a few), that are mastering the R&D integration.

The research and academic community is introducing amazing ideas and new approaches, pushing the limit of human knowledge. However, in the particular case of Text Analytics research, several (if not most) of such ideas are never implemented in a real commercial environment. There are several potential reasons for this:

1. A solution that worked well on small scale datasets might not be able to scale to the amount of data processed by a specific business in a given period of time. Alternatively, the solution might be able to scale, but at a high cost. Most of the times, models can be scaled-up be adding more computational power. However, this could make the solution more expensive than the perceived benefit of using it. For instance, imagine a model that cost twice as much as the current approach and that provides 1% improvement on quality. This approach is not likely to be chosen in a real-life scenario.

2. The problem itself does not need a solution. Just to be clear, I am not saying that research should focus only on problems born in a commercial environment. If we would have done that, some of the most important discoveries in history might have been lost. Nonetheless, we have to understand that some research has not real application at the moment of its conception.

3. The solution is difficult to implement or maintain. Although some people like to see algorithms and models as “magic” black boxes, this is usually the opposite of what software engineers and architects need. They need to understand the implementation principles of such libraries in order to do their job better. Unfortunately, this exposes one of the main drawbacks of the academic community: A lot of researchers are not that good at coding. This is illustrated in research libraries which cannot be used in a commercial environment because they lack a minimum level of developing quality. This could be seen on poorly defined APIs, lack of multi-thread capabilities, incorrect documentation, outdated technologies, complexity to deploy, …

In addition to these problems, even if a scalable, good-enough quality is implemented in a correct way, the integration of such module in a production pipeline might present additional challenges in itself.

I am afraid that I have not found a silver bullet to solve these problems. However, we have came up with a process in Signal that has shown to minimise some of the problems for us. For every new component into the system (e.g., machine translation) we follow the S.I.M.P.L.E. steps: Study, Implement, Model, Prepare, Live, and Evolve.

This step involves catching up with the latest developments in the research community. Specifically, we would focus on the following points:

  1. How is the problem defined
  2. How is the quality of a solution measured
  3. What are the best approaches to address it and how good they are
  4. Common mistakes to avoid and quick wins (e.g., simple ideas that seem to perform well)
  5. Current libraries or services

Once we have done the literature review, it is time to implement the first ever solution to the problem, and in order to do so we will focus on the simplest possible approach that could work. This Proof of Concept (POC) will allow to see the potential benefit of the component for our system. After the POC is completed, a qualitative evaluation is perform by manually observing the output of the system based on a small sample of our own data.

Assuming that the POC showed some potential, we move into the modelling phase where a proper framework for the problem is created.The main goal is to allow the automatic evaluation of multiple models with one or more static datasets. This aligns directly with the traditional experimentation setting in most of the related research fields. This could also include data collection and annotation. Obviously, the evaluation metrics are modified to fit our business goals. This code should be of high standard and it must have good test coverage and design. At the end of this step we should be able to rank and compare different approaches both by effectiveness and efficiency.

If at least one of the models selected from the previous step is potentially good enough from the business perspective, they are moved into the preparation phase. At this stage, people from research and development refactor the code-base to make it more efficient, elegant and maintainable. They usually increase the test coverage and documentation as well. Moreover, architectonical questions related to the integration of the component are also answered in this step: What extra infrastructure do we need? Can we use the current data model? How and where do we store the models? Can we cache some of the data? … These are only a few of the multiple questions we usually face in the discussion. After all this is done, the model, or the whole framework in some cases, is ready to go “live”.

The development team integrates the framework or selected models into the pipeline and the new version of the system is pushed into a staging environment. Everything is now ready to analyse new data being processed by the new component and to allow a front-end feature to use this information.

The last, but not least, step of the process is to keep evolving and tuning the solutions over time. Topic drift and changes in the environment or the product will have a massive impact of the model itself and therefore, its quality might decrease. This step requires monitoring tools to inspect the data and the system looking for changes, as well as continuing to research new ideas to solve the problem.

We believe that this steps allow an almost seamless integration between the research and development teams and improve the research capabilities while allowing for quick deployment of new solutions or components. It also helps the cohesiveness and increases the knowledge of the teams by requiring a high degree of interaction between researchers and developers.

I believe that research and development integration is one of the main challenges in innovative companies within the analytics space and I have presented one of the processes we have in Signal to address it. However, there is no silver bullet and the SIMPLE approach might not work for all companies. In our case, both researchers and developers have embraced it and are quite happy with it.

OKRs (Objectives and Key Results)

This blogpost is focused on one very important part of any business: Align the goals of the company with each team and the personal objectives. One of the techniques to address this challenge in an elegant and productive way is known as OKRs (Objectives and Key Results). OKRs have been used before in large companies such as Oracle or Intel, but they are probably better known because Google has adopt them as a core part of the organisation.

Continue reading