SEGES Innovation creates the future of agriculture with AI and MLOps

Photo: SEGES Innovation

Summary: What SEGES Innovation has achieved

  • A reliable and fully automated AI platform based on twoday kapacity’s Best Practice MLOps Framework that streamlines production setup and ensures high quality

  • Time spent on maintenance and retraining of machine learning models is reduced to practically nothing

  • Costs are reduced by 80%

  • The solution frees up time for data scientists to focus on what’s important: creating smart and innovative data science products that create value for users

An innovative Research & Development organisation

SEGES Innovation researches and develops solutions for the food and agricultural sector of the future. For more than 40 years, they have worked to bridge the gap between research and practical experience to improve agriculture and reduce its impact on the environment.

Today, part of this work takes place at SEGES Innovation’s data science team. They work, among other things,  on exciting projects such as predicting diseases in cows, auto-accounting and predicting crop yields from satellite data.

Resource-intensive to maintain models

To train and put machine learning models into production, SEGES Innovation had a solution that was partly on-premise, used a lot of different tools and had a large manual code base.

This meant that the data science team spent a lot of resources on maintaining and retraining the models, and that there was a long way from model development to production.

“We spent many hours maintaining and retraining the existing models. That’s why we wanted to build an AI platform based on best practice MLOps principles. By focusing on automated workflows, we have been able to free up a lot of time, which we have instead spent on developing new data science products that can create value for users,” says Lasse Rose Malskær, Lead Data Scientist at SEGES Innovation.

Therefore, they decided to implement a new AI platform with a focus on minimizing maintenance time. They would achieve this by:

  1. Prioritizing simple solutions
  2. Outsourcing maintenance of the platform to PaaS
  3. Automated processes rather than manual processes

“We had the opportunity to start over and rethink how we could build a new AI platform from scratch so that it suited our needs. Here, Microsoft Azure (Machine Learning) was the optimal solution for us, and we chose twoday kapacity as a collaboration partner due to their extensive experience with Azure,” Lasse Rose Malskær says.

Photo: SEGES Innovation

Streamlined processes save resources

Over the course of three months, twoday kapacity and SEGES Innovation moved the entire existing setup to the new platform. Along the way, twoday kapacity further developed their MLOps Best Practice Framework with the learnings made along the way.

The work was based on a tangible model that was in demand by stakeholders as the focus was on delivering value to the business continuously with the transformation. Subsequently, the rest of the platform was built around it, and the remaining models moved to the new environment.

The result is a significantly more streamlined platform, where the infrastructure is set up as infrastructure as code. It has become significantly easier to start a new data science project, since everything is defined in advance in a template. This means that everything from CI/CD pipelines and code to directory structure has been streamlined.

“That saves us a lot of time. We can go from POC to production much faster than before, so we get something out to users faster. At the same time, the template ensures that all projects from the start have predefined automatic CI/CD/CT workflows, the necessary infrastructure such as code and a uniform and good code practice,” Lasse Rose Malskær says.

Before:

After:

With the new solution, SEGES Innovation has a simpler setup, which makes it significantly easier to launch new data science projects.

What is MLOps?

MLOps (ML Operations) is a core feature of machine learning engineering that focuses on streamlining the process of putting machine learning models into production and then maintaining and monitoring them.

But why do we need it?

As with any other software system, machine learning applications must be deployed and monitored to be useful and provide value to businesses.

We know this from traditional DevOps, which is used to keep track of:

  • Version control
  • Testing of code
  • Continuous Integration and Delivery (CI/CD)
  • Application monitoring

But machine learning applications have additional complexities, which is why an MLOps framework is necessary:

  • In addition to code, we also need to version and track machine learning artifacts, such as an Machine learning model, as well as training data
  • Machine learning is less linear and more experimental
  • Testing a machine learning model is more difficult due to the statistical nature of machine learning
  • Even without code changes, the systems must be periodically trained and tested with new data

Photo: SEGES Innovation

Versioning and automatic tests ensure good governance

In the current MLOps setup, everything is versioned; training, the code, data in the different steps, the models. Lasse and his team can follow this all the way through the pipeline, and should an error occur, they can go back and see where the error occurred.

In addition, together with stakeholders, they have defined a series of tests where retrained models are automatically tested against the existing models. This enables the retraining and deployment of models to run completely automatically without interference from Lasse and his team, who simply receive emails about whether everything has gone according to plan.

“We hardly spend any time retraining and deploying models, which before could take up to a week depending on when stakeholders had time. By automating this process, we have not only freed up a lot of time, but through champion/challenger tests we also automatically ensure that it is always the best models that are in production by the users,” Lasse Rose Malskær says.

A cost reduction of 80%

The data science team at SEGES Innovation has reduced costs by 80%. However, this is also largely due to the team’s work in making better and more simple models that are not as resource-intensive as before.

Lasse and the rest of the team have been enthusiastic about how the AI platform has made their everyday life significantly easier. The collaboration with twoday kapacity has proceeded based on twoday kapacity’s co-build approach, where the solution is built together, and where twoday kapacity’s consultants continuously train SEGES Innovation in using the solution. And the collaboration has been a success.

“The collaboration with twoday kapacity has been a real pleasure. It has felt like bringing in some enormously competent colleagues who have not only co-delivered the overall plan, but have also taken responsibility and worked on an equal footing with us in the team,” Lasse Rose Malskær says.

Photo: SEGES Innovation

About SEGES Innovation

SEGES Innovation P/S is a private, independent, non-profit research and development organisation.

SEGES Innovation carries out research and innovation tasks within agricultural and food-related areas such as plant production, livestock production, environment, climate and bioeconomy/bioenergy, circularity, biodiversity and economy.

In addition, the organization develops, maintains and supports many of the farmer’s other IT tools – for example, the programs the farmer uses to manage operations in cattle, pigs or plant breeding.

SEGES Innovation employs around 530 people.

Læs mere om SEGES Innovation her