MSC Internship assignment: " Reinforcement Learning on AI sail case study"
Wageningen, Gelderland
For MARIN Academy we are looking for a student for the following MSc internship/assignment
(6 months):
MSc internship - Reinforcement Learning on AI sail case study
Duration 6 months
Location In the office of MARIN, Haagsteeg 2, 6708 PM Wageningen, the Netherlands.
Company MARIN – Maritime Research Institute Netherlands – is a globally recognised and independent institute for maritime research. Our mission is 'Better Ships, Blue Oceans': we stand for clean, smart and safe shipping and sustainable use of the sea. We do this as an independent knowledge partner for the maritime sector, government and society. We have a complete range of facilities: model test basins, in-house software, simulators, numerical facilities and measurement techniques to test, simulate and monitor ships and operations. The unique combination and synergy of these facilities and tools enable us to be involved in the entire lifecycle of AI projects applied to maritime domain and to achieve an optimal sim-to-real transfer. More info: Maritime Research Institute Netherlands | MARIN
Project background One year ago, MARIN’s AI Sail team took up a challenge: can a computer learn to sail an Optimist with the help of AI? Friday 24 November 2023, was the moment of truth, during a demonstration with an Optimist in our Offshore Basin. Most maritime prediction methods are based on a model-based approach: physics-based models are combined in a computational model and validated in model tests and reality. With AI Sail we want to demonstrate the possibilities of data-driven methods, where the physics are not explicit in the model, but implicit in the data. When applied in the real word, AI often suffers from a degradation of its performance learned in a simulation. With AI sail project, we were able to address this issue. Thanks to MARIN complete range of facilities, we were able to create the condition for an AI to train in a simulation before entrust it with the Optimist control. The AI Sail team consists of a broad mix of MARIN specialists: machine learning, time domain simulations and model testing. The AI designed for this project is a reinforcement learning (RL) agent which learns to optimize its policy through interactions with a dynamic environment. The virtual environment is a time domain simulation of the Optimist and our Offshore Basin powered by our XMF framework. The training took place in our computational cluster. In the meantime, our model test engineers had modified our Optimist with remote-controlled rudder, sheet and ballast. In November, while discovering the basin for the first time, the AI managed to transfer the strategy learned in the simulation to sail the real Optimist upwind without further training. This experiment has proven the ability of our AI to maintain its performance in transitioning from a simulated environment to the real world.Moreover, training in our simulation has several benefits:
Easier development by running parallel trainings with various hyper parameters,
Safer training by allowing the agent to experiment extreme conditions without any risk of ship damages or crew injuries.
Speed-up training thanks to parallelization of parameters fitting and fast time simulation.
Role & tasks We want to take advantage of AI sail framework to explore and compare more reinforcement learning variants such as offline learning, model based, hybrid control, physic informed, curriculum learning, evolutionary strategy, etc. The work will be divided into the following tasks: 0. Select one RL variants based on your interest, experiences, discussions with your mentor. 1. Conduct a literature study and Python libraries review on this RL variants. 2. Develop an add-on to MARIN reinforcement learning framework. 3. Build a benchmark procedure on AI sail case (environment settings, task, evaluation metrics, statical analysis…) to objectively compare RL variants. 4. Conduct an evaluation campaign both on the benchmark RL and selected variant. 5. Deliver a well-commented and documented code both of the benchmark procedure and RL variant. 6. Present the findings to the Data Science team of MARIN. 7. Synthesize the results of the project, insights, and recommendations into a thesis or report.
Work environment The trainee will be supervised by a machine learning specialist from R&D department. MARIN provides a personal workstation with access to the cluster and software licenses needed for the mission. The internship will be done in an international and multicultural environment. Additionally, students benefit from a dedicated area with access to amenities (canteen, rest room, shower, etc).
Requirements Skills and experiences expected from the candidate:
Must have: good Python skills, experience with Object-oriented programming (OOP), knowledge on reinforcement learning.
Good to have: experience with Git, Linux, sailing, code development, statistical tests.
Language English
If you are interested, Fanny Rebiffe, Applied Data Scientist can tell you more about the internship: f.rebiffe@marin.nl. You can apply for this internship by using the APPLY button.