To test it out, install VSCode and the OCaml Platform extension. Having them both installed will automatically open the walkthrough. Should you need to manually open the walkthrough after installing the OCaml Platform extension, start by clicking on the help menu in the top right corner, then select ‘open walkthrough’, and select the one titled ‘OCaml: Setup opam dev environment (manual)’.

Mission

VSCode is by far the most popular editor among beginners, mainly thanks to its helpful UI, which gives users plenty of support: welcome/setup panels, interactive menus, selection tools, and so on. By targeting VSCode first we are making installation easier for newcomers, who are more likely to struggle to set up OCaml. It will also benefit anyone who prefers the VSCode editor and provide maintainers with helpful feedback to improve other workflows.

Our mission at the start of the project was to focus on reducing friction by minimising reliance on the terminal and external setup documentation. One of the earliest pain points when installing OCaml is understanding where to find and install everything you need to get started with writing code. Even if you know where to find everything, it can be challenging to get to it all in the right order. OCaml.org has an installation guide, which is a great resource, but it’s a lot of information to take in and can be hard to parse.

We care about growing the OCaml community, and feedback suggests that installation is a common pain point for newcomers. To improve their experience, we need to ensure that the pathways to installation and setting up projects are smooth.

The Walkthrough

The new VSCode walkthrough opens a window when users install the OCaml plugin. The page guides the user through five steps, providing clear instructions and a UI that indicates when each step is in progress and when it succeeds. When the user clicks on a step, the walkthrough opens an integrated terminal and completes it for them.

The steps are:

• Install opam: VSCode opens a terminal and runs the opam install script.
• Initialise opam: VSCode opens a terminal and runs opam init. Initialising opam prepares the user’s system to use opam to manage OCaml packages and compilers.
• Activate the opam switch: This step prompts the user to select and activate the opam switch. It explains that an opam switch is an isolated OCaml environment where users can install different versions of OCaml and packages.
• Install Platform Tools: The next click installs some key OCaml tools: OCaml LSP for editor support, odoc to generate documentation, ocamlformat to format code, and utop as an interactive REPL.
• Check Installation: This step verifies the OCaml installation by running utop.
• Congratulations: Success! OCaml is installed, and users are asked to fill in an optional feedback form.

Here's what the start screen for the walkthrough looks like:

Until Next Time!

There are still things that the team is hoping to iterate on in the coming months. Using the input from the feedback form, the team wants to improve the walkthrough as well as simplify the documentation on OCaml.org to reflect the new changes. Once Dune Package Management has its first full release, the team also plans to create a similar walkthrough for that workflow to support its users.

You can connect with us on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects. We look forward to hearing from you!

Now, with a new library called ciao-lwt, users can automate part of the migration process from Lwt to Eio. One of our engineers, Jules Aguillon, has been developing the library and using it for the Ocsigen project. This post shares his work and will introduce you to ciao-lwt, how to try it, and what limitations you should expect.

The project is made possible thanks to a grant from the NLnet Foundation, which funds research and development projects furthering internet technologies and the open internet, and the NGI Zero Core fund of the European commission.

Why Would I Switch to Eio?

Ultimately, the concurrency library you choose comes down to a matter of taste, but Eio has some nice characteristics that you may find worth the switch. Since it is direct-style, Eio does not require you use a monad for concurrency, which gets rid of the so-called ‘function colouring problem’. The resulting code is faster, less complex, and has some nice security capabilities.

You can read our blog post on Eio 1.0 for more context.

Ciao-Lwt

The library contains a collection of tools for translating an Lwt library into Eio code. Lwt marks concurrent code and non-concurrent (sync and async) code using bind operators or bindings, and functions must be explicitly marked as sync or async for the program to run. These are the ‘bind’ and ‘map’ operators, including Lwt.bind, Lwt.map, let*, let+, as well as the infix operators >>= and >>|.

The first step in turning Lwt into Eio code is to get rid of the bind operators. They weave through every part of Lwt code and are time-intensive to remove one by one. Ciao-lwt can automate the process and remove the bindings for you. However, the library has some limitations, which will be explained in more detail below.

At the moment, the library contains the following tools:

• lwt-ppx-to-let-syntax: Removes instances of lwt_ppx and replaces them with Lwt library function calls,
• lwt_lint: Finds implicit forks in Lwt code,
• lwt-log-to-logs: Rewrites files containing Lwt_log and migrates them to use Logs,
• lwt-to-direct-style: Finds and rewrites Lwt and other Lwt modules, turning them into Eio code instead.

Lastly, something to bear in mind is that ciao-lwt uses Merlin's index to locate every use of Lwt.

How Do I Try It?

To get started with ciao-lwt, the first thing to do is visit the repo and install the tools in the opam switch you’re using to build your projects using the command:

opam install ciao_lwt lwt_lint lwt_ppx_to_let_syntax

To make reviewing the change easier, make sure your code is formatted. The tool will entirely reformat the file it touches, which may make actual changes harder to see.

The first step is to remove any use of lwt_ppx (for example the let%lwt syntax):

lwt-ppx-to-let-syntax .
dune fmt # Remove formatting changes created by the tool

This operation is purely syntactical, the tool simply walks the given directory tree and parses every .ml files it finds, updating the files that contain usages of lwt_ppx.

Before running the next tool, try eliminating common causes of implicit forks:

lwt-lint .

This operation is also purely syntactical. The tool warns about every occurrence of let _ = .. and ignore that doesn’t have a type annotation. This helps you find cases where an Lwt promise is disregarded. To silence each warning, add a type annotation, for example: let _ : my_t = .. and ignore (.. :my_t).

If you use Lwt_log, you can migrate to Logs easily with:

dune build @ocaml-index # Build the index (required)
ciao-lwt to-logs --migrate .
dune fmt # Remove formatting changes created by the tool

This tool works similarly to ciao-lwt to-eio described below. It is provided as a separate command because your program will likely work as before but it lets you review this step independently and it simplifies the next step.

Finally, migrate to Eio:

dune build @ocaml-index # Build the index (required)
ciao-lwt to-eio --migrate .
dune fmt # Remove formatting changes created by the tool

This operation migrates the common uses of Lwt, but the transition is not yet complete.

Limitations & Considerations

Ciao-lwt is still considered experimental and a work-in-progress, which you should bear in mind when you try it. Your feedback and input is very welcome and will help the team improve the tools.

It sounds obvious, but as a promise-based concurrency library, Lwt creates a lot of promises. Everything that is concurrent in Lwt is a promise; it specifies actions that will happen at a later time. Some promises are so-called ‘implicit forks’, which do not use the bindings we mentioned earlier.

Let's look at an implicit fork:

let _ =
  let a = operation_1 () in
  let* b = operation_2 () in
  let* a = a in
  Lwt.return (a + b)

Here, let a = operation_1 () in 'forks', meaning it creates a concurrent thread. Since there are no binding operators or Lwt function calls, ciao-lwt can't detect this fork syntactically or with Merlin's index.

As a result, while Lwt would run operation_1 and operation_2 concurrently, after ciao-lwt converts it to Eio it would instead run sequentially:

let _ =
  let a = operation_1 () in
  let b = operation_2 () in
  let a = a in
  a + b

Users need to be aware of how ciao-lwt handles 'implicit forks' so they can fix bugs introduced in the migration.

The most helpful tool to verify your new code is the OCaml compiler. Your resulting code will likely not typecheck, and OCaml's typechecker can guide you towards the manual changes you will need to make. It's not foolproof, and you will still need to be on the lookout for concurrency bugs, but ciao-lwt's tools in combination with the OCaml compiler will give you a nice head start on your journey from Lwt to Eio.

Until Next Time

Tarides remains committed to creating new tools that make new and old workflows easier. We hope ciao-lwt proves useful to you, and appreciate any feedback you have to share.

You can connect with us on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects. We look forward to hearing from you!

WSOO is already known for its speed. Users switching from Js_of_ocaml (JSOO), which translates OCaml bytecode to JavaScript, can expect significant performance improvements. Back in early 2025, Jane Street reported that they saw significant improvements using WSOO in comparison to JSOO. With the most recent optimisations WSOO is even faster!

You can try Wasm_of_ocaml for your own projects by visiting the manual, which includes installation instructions, on the Ocsigen website.

How Have We Made Wasm_of_ocaml Faster?

Almost as soon as the feature-complete release of Wasm_of_ocaml was out, a team at Tarides began working on optimisations for the library. Jérôme Vouillon has led the way, testing each change and measuring the resulting performance improvements. Let’s take a look at some of the PRs that have come out of this effort and what has changed:

• Inlining pass: Jérôme rewrote the inlining pass to adjust its behaviour in certain cases. The change avoids inlining code in a loop that WASM engines can’t optimise because V8 currently has no way to switch to a more efficient code while executing a loop. It is now also more assertive in inlining functors and functions, including List.fold_left. The PR is #1935.

• The OCaml standard library contains a Bigarray module that offers an API to manipulate an array of numerical values (integers of floating point values of various sizes). When compiling a program to WebAssembly, Bigarray operations are translated behind the scenes as operations on Javascript “typed arrays”.Up to now, these operations were implemented in terms of direct access to these arrays, but this method went through calls to Javascript functions that incurred some overhead.

  Javascript offers an alternative API called DataView, whose operations are recognised and compiled as direct memory accesses by Wasm engines such as Google’s V8. We have modified the backend function supporting Bigarray to use DataView and benchmarked it with a program performing millions of Bigarray accesses, and we are observing an impressive 3.9x speedup.

  Check out PR #1979 to look more closely at the changes.

• Function call optimisations: There are a whole host of changes targeting function calls:

  • PR #2041 optimises the representation of closures using more precise types.
  • PR #2044 optimises calls to a statically known function.
  • PR #2059 omits the code pointer when it is not used (because of the previous optimisation). This reduces the amount of memory allocated by the program, but also allows Binaryen to perform some global optimisations.

• Integer optimisations: Wasm_of_ocaml uses 31-bit integers to allow for a uniform representation of integers and references. The integers have to be converted to 32-bit integers to perform numeric operations. PR #2032 optimises this workflow by avoiding unnecessary conversions between 31 and 32-bit integers, which speeds up performance.

• Number unboxing: Avoids boxing numbers (both within functions #2069 and outside of functions #2101) when the boxed value is not used. The change significantly improved the microbenchmarks almabench and fft.

• Number comparisons and bigarray operations: Specialisation of number comparisons and bigarray operations in PR #1954, based on a type analysis, optimises the performance of functions. Future work to improve on this optimisation centres around using hints from the OCaml compiler.

Hopefully, you now have a good sense of just how much has been tweaked and improved to make WSOO faster! But, you may ask, just how much faster are we talking about?

Benchmarks

Several of these changes have contributed to improving WSOO's performance, visualised in the graphs below. The first one is the CAMLboy benchmark, showing steady improvement over time until it was one third faster.

The second graph show the performance improvement for some selected microbenchmarks. Numerical benchmarks, like almabench, fft, nucleic, and raytrace, show a nice improvements. The integer optimisation helps for fib, quicksort, and fft. The raytrace microbenchmarks show the most improvement, thanks to better inlining and number unboxing across function.

What’s Next for Wasm_of_ocaml?

Look out for the next milestone for WSOO: adding WASI 0.1 support! The team have implemented an alternative runtime based on WASI, a group of API specifications for software compiled to the WebAssembly standard (W3C). WASI support will enable users to execute OCaml programs in new environments, from browsers to clouds to embedded devices. The PR is still open and feedback is always welcome.

Furthermore, another one of the team’s goals is to test an implementation of effect handlers based on the Stack Switching proposal, you can find the details in #1882.

Lastly, the team has more work planned to improve the performance of both Js_of_ocaml and Wasm_of_ocaml in 2026.

Until Next Time

Remember to visit the manual on Ocsigen’s website to learn more about Wasm_of_ocaml and to get started with the compiler if you haven’t already. The WSOO team is always keen to hear feedback and figure out how they can improve the user experience. Please share your thoughts and experiences with the compiler on OCaml’s discussion forum!

You can connect with us on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects. We look forward to hearing from you!

What is odoc?

Briefly, odoc is a documentation generator and rendering tool. It can read doc comments from source files and transform them into formats such as HTML, LaTeX, or manual (‘man’) pages (as of 3.1, odoc also supports an experimental markdown backend). Furthermore, it enables cross-referencing, based on an extended version of ocamldoc markup, allowing you to create links for functions, types, modules, and documentation pages. Documentation generated by odoc can include links to the source code of functions, making navigating between the documentation and the code easier for the reader. The tool also automatically highlights syntax in code snippets to make documentation more legible.

You can learn more about odoc on its website and on its OCaml.org page.

What’s New With odoc 3?

3.0 was a big release, bringing multiple new features to the documentation generator. The main new features that came with the update are:

1. Search by type: Sherlodoc now lets users complete several specific searches in odoc documentation. These searches include searching by name, searching for constructors of a type, using _ to omit a subtype and search for consumers of a type, plus several more specific search options. This makes it easier for users to find lost values and navigate the docs. Previously, Sherlodoc was a separate project, but now it is integrated into odoc’s code base.
2. Global sidebar: odoc can now generate global sidebars and breadcrumbs for pages. Previously, pages could ‘hack together’ their sidebars and breadcrumbs (this is what OCaml.org was doing), and now it can all be managed through one tool. The change offers greater choice for authors to organise and present documentation more clearly with minimal fuss.
3. Support for Manuals: odoc 3 comes with several features that help users create mld pages. These mld pages are written using odoc’s markup language. For example, the update gives users the ability to use hierarchical manuals and to link to the manual pages of other packages or API docs for libraries that aren’t in their direct dependencies.
4. odoc Driver: This driver helps users manage some of the more involved settings and features when running odoc. For example, rendering source code is much easier than when the feature was first introduced in odoc 2, since odoc_driver now manages it for you.
5. Generated documentation and source code: You can jump straight from items in docs to the source code using a ‘source link’, which navigates readers to a rendered version of the source code, without creating a custom driver.
6. Media: Images, videos, and audio files can be added to documentation generated by odoc. This will make tutorials and documentation more intuitive and user-friendly, offering more ways to learn and share knowledge. Check out this OCaml.org tutorial to see images using odoc in action!
7. Cross-package links: Now, odoc lets you use cross-package links in documentation, meaning you can reference any other package's modules and pages on a doc page. It allows users to generate interconnected pages of documentation to help readers navigate intuitively and smoothly.
8. Support for incremental build systems: The update makes managing build dependencies easier by supporting incremental build systems and allowing for better shared build caches. In the future, when odoc 3 rules have been implemented in Dune, the integration with Dune’s cache will make this feature especially useful.

OCaml.org and odoc 3

Once odoc 3 had been released, the OCaml.org team started working on upgrading the package docs section of the site to use the new version. Since the tool underpins the entire documentation pipeline of the OCaml community, it was important to get buy-in from them and give them a chance to review the proposed changes ahead of time.

So, the team shared updates on OCaml Discuss with links to the ‘staging’ version of OCaml.org, where users could explore the changes and share their feedback. It allowed them to help discover bugs and discuss how they felt about the implementation, which gave the maintainers crucial information to make the transition as smooth as possible. Previewing changes on the staging website also helped ensure the update did not break any existing features.

When the team were updating the docs pages, they needed to consider the significant changes to the CLI and new driver that odoc 3 had made, as well as the new feature of linking to other packages’ docs. They also adjusted the build and caching method employed by the documentation pipeline to improve support for incremental builds. With these background changes done, the website was ready for its makeover! You can check out PR #3124 to look at the patch.

Although updating OCaml.org to use odoc 3 was good for users, since they could take advantage of the new features, it was also useful for authors who could explore the tooling before they used it themselves. For example, seeing how global sidebars, breadcrumbs, and media are now implemented through odoc and supported on the website.

Tarides and odoc

Part of Tarides' mission is to encourage new users to adopt OCaml by making it easier to learn and use the language in their projects. Improving odoc is a quality-of-life upgrade for developers that makes creating documentation easier. In turn, this benefits other users by hopefully encouraging more documentation to be generated.

In the same vein, we are also committed to maintaining and improving OCaml.org as a resource for the entire community. This includes upgrading it to use the latest versions of tools like odoc and testing and implementing new features.

OCaml is a language with many strengths, including its secure-by-design features. We want to give as many new people and organisations access to the right resources to make using OCaml easier. By making languages like OCaml mainstream, safer and more efficient code becomes more prevalent.

Try odoc 3 for Your Documentation!

You can try odoc 3 for yourself and experiment with the new features. The readme and installation instructions are on OCaml.org, which is an excellent place to get started. The maintainers of odoc welcome any feedback and input, either on Discuss or directly in the GitHub repo.

You can connect with us on Bluesky, Mastodon, Threads, and LinkedIn or sign up to our mailing list to stay updated on our latest projects. We look forward to hearing from you!

In this post, we provide a brief overview of the relationship between Merlin, LSP, and Neovim in OCaml, the goals and features of ocaml.nvim, and offer some insight into how the engineers approached this project. We encourage you to install the plugin if you’re a NeoVim user (or just curious!) and give it a try. The repository is public and comes with documentation about its features and how to install the plugin. To help the team out, you can share your feedback on the OCaml Discuss forum, in the repo, or by emailing Charlène directly.

Merlin, LSP, & NeoVim

The Language Server Protocol (LSP) was created to address a programming landscape of a growing number of incompatible editors. The open protocol standardises the interactions between an editor and a server providing IDE services, creating a standard model supporting any compatible editor.

Created before the LSP, OCaml’s editor-agnostic server, Merlin, has offered advanced in-editor functionalities to the language for a long time. It quickly gained popularity within the community and was supported by several editors, including Emacs and Vim. However, supporting and maintaining each individual editor was resource intensive, as they required a tailored approach whenever they, OCaml, or Merlin updated.

Now, thanks to the fact that most modern editors use LSP to communicate with programming languages, OCaml can use LSP defaults to integrate Merlin with each editor’s LSP plugins. Neovim is one of the most popular editors (alongside Emacs, Vim, and VSCode) in the OCaml community. The new ocaml.nvim extension enables Neovim to support custom commands that are not possible with the standard LSP. The plugin ocaml.nvim works with generic Neovim LSP to provide access to advanced ocaml-lsp features, including all the advanced Merlin commands not supported by generic LSP clients. This approach addresses the so-called ‘editor burnout’ problem, which we have described in greater detail in our post on ocaml-eglot, ocaml.nvim’s ‘sibling’.

OCaml.nvim: Creating a New Plugin

The goal of the project was to create a modern and idiomatic Neovim plugin that would implement the server’s custom requests in a way that respected the design of the editor. We wanted to create a good foundation for community and industry users to build on, with a minimalist and well-documented plugin.

The ocaml.nvim plugin, of course, also supports key custom OCaml features, including advanced type-enclosing functionality, syntax-aware navigation, search by type, construct, and switching between the corresponding .ml and .mli. Check out the ocaml.nvim repository to read more about the supported features.

Code Actions and Custom Requests

Since this plugin aimed to enhance the standard LSP features, we needed to rely on an additional protocol to link specific information to a particular command. LSP offers two types of special requests: code actions and custom requests.

The LSP already lists code actions, but when there are numerous options, this special request can be confusing to use. Moreover, code actions prevent the use of parameters. They do not, for example, let the user select between multiple choices. This is where custom requests come into play.

Custom requests allow the language server to converse with the user as much as needed, for example, to collect choices. As a consequence, they require an extra plugin to translate the conversation from the server to the user, as the LSP cannot generalise the infinity of commands we could imagine in all languages.

Every function is available on the Merlin side and accessible via ocaml-lsp. There are two types of calls.

• Ocaml-lsp API: This is the language server for OCaml. It provides the default features to the LSP. It also offers more than that. We can connect to the API to request information based on the cursor location or the server's knowledge of the current file.
• MerlinCallCompatible: This is also provided by ocaml-lsp, and allows us to invoke the Merlin commands. It requires additional arguments and returns the result in a character string format; here, we used JSON.

UI Elements

Our objective was to provide a clear interface. To achieve this, we implemented four main UI elements for the entire plugin.

• Selector: opens a floating window where the focus is initialised at the first line. Then, you can use the arrows and enter to move and select the wanted line.

• New Window: This is built-in with nvim, and splits the current window to open a new buffer.

• One-line Display: This is the simplest UI, simply a print. If it takes less than one line, it is displayed where you enter the commands.

• Multi-line Display: By default, if the string in print exceeds one line, it displays it in multiple lines. However, this behaviour is not ideal because it requires pressing Enter to erase the text in an unappealing manner. So, the plugin provides an alternative that is similar to the UI selector.

Regarding customisation, since everyone has their own keymaps, we provided default ones and allow users to modify them to suit their needs and preferences.

Try it Out!

Visit the ocaml.nvim repository on GitHub to get started with the new plugin. The repo includes the installation instructions, a straightforward process using lazy.nvim, as well as a features list and some examples.

Since the team is working towards a 1.0 release, any and all feedback is welcome. Remember to share your thoughts, suggestions, and questions on Discuss or in the repo through issues and commits.

You can connect with us on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects. We look forward to hearing from you!

If you didn’t get the chance to go this year, or if you just want to relive it, this post has you covered! We’re going to take a look at everything OCaml at the conference – from OxCaml to improved editors and establishing a community code of conduct – and hear first-hand from some of our engineers who attended.

Next year’s ICFP is in Indianapolis, and you should keep an eye on the website to register and submit your papers.

Experience Reports

Several of my colleagues attended ICFP, and just like last year, I asked them a bunch of questions about their experience when they returned! Firstly, I asked them all why they enjoy ICFP and why they choose to attend it year after year. David Allsopp highlighted the social networking opportunities: “For those of us lucky enough to be able to attend them, conferences grease the social gears of collaboration”. As an example, David mentioned how a talk on range analysis in Standard ML inspired him and Ryan Gibb (who incidentally work next door to each other in Cambridge) to start hacking on package-management solving right then and there! Read more about David’s time at ICFP in his blog post.

Sudha Parimala explained that “ICFP has many different tracks that are highly relevant to Functional Programming in general and also to OCaml. It's a good place to meet folks working on similar things and discuss ideas.” In addition to the various tracks, ICFP also hosts events that provide opportunities to meet new people. “Even though I couldn't make it this time, I'd highly recommend the ‘Women in PL’ dinner hosted at ICFP. It gives you a good platform to connect with other women programmers and researchers.” If you’re attending ICFP next year, keep an eye out for these organised events.

Speaking of next year, the submission deadline for papers is on February 19, so, in the words of Sudha: “If you're working on cool OCaml stuff, consider submitting it to the OCaml workshop next year!” and we look forward to seeing you there!

The OCaml Workshop and Talks

Let’s dive into everything OCaml (and some exciting honourable mentions!) that happened at ICFP this year. Where possible, I’ve linked the recordings of the talks so you can recreate the conference experience from home!

The OCaml Workshop was great, as usual, and took place on October 17. The following talks were presented by developers of many different affiliations, including IIT Madras, NIT Trichy, Tarides, the University of Illinois Urbana-Champaign, Jane Street, Bloomberg, and the University of Cambridge.

OCaml Workshop Talks

• A Mechanically Verified Garbage Collector for OCaml by Sheera Shamsu, Dipesh Kafle, Dhruv Maroo, Kartik Nagar, Karthikeyan Bhargavan, and KC Sivaramakrishnan. OCaml’s garbage collector (GC) is crucial to the functioning of its runtime system and overall correctness and safety. This talk shared details about a project to create a correct, proof-oriented GC that can evolve with the language over time.

• OCaml Package Management with (only!) Dune by Stephen Sherratt, Marek Kubica, and Rudi Grinberg. This talk showcased how developers can now use Dune to download and install packages without relying on any other tool (like opam).

• How the OCaml Community Established Its Code of Conduct by Sudha Parimala. As Sudha said, “OCaml has had a thriving open source community for almost three decades now, but a code of conduct was only introduced in 2022.” Her talk described the process behind establishing that code of conduct, including creating the team, gathering feedback, and lessons learned.

• Embedding WebAssembly in OCaml for Safe Program Construction by Hunter DeMeyer. This talk proposed WasML: a library enforcing syntactic and semantic constraints in wasm programs via OCaml types.

• smaws: An AWS SDK for OCaml by Chris Armstrong. Introduced a new Amazon Web Services-based software development kit for OCaml, smaws, exploring the challenges and design choices behind the new library.

• Toward a More Secure OCaml Ecosystem by Maksim Grankin. All about the new OCaml Security Team, launched by the OCaml Software Foundation (OCSF), the motivations behind creating the team, its goals, and the security challenges facing language ecosystems today.

• Three Steps for OCaml to Crest the AI Humps by Sadiq Jaffer, Jonathan Ludlam, Ryan Gibb, Thomas Gazagnaire, and Anil Madhavapeddy. The talk looked at how well-represented OCaml is amongst open-weight models and what could make it stand out, outlining potential changes the ecosystem can make to work better with AI coding assistants.

  In his blog post about the talk, Sadiq described how “The gap between coding agent performance on mainstream versus niche languages could prove fatal to smaller language communities. New developers increasingly judge a programming language not just on its traditional tooling (compilers, debuggers, libraries) but also on how well AI coding agents support it. If agents struggle with OCaml, fewer new developers will choose to learn it, creating a vicious cycle.”

• A New Era of OCaml Editing: Powered by Merlin, Delivered via LSP by Xavier Van de Woestyne, Sonja Heinze, Ulysse Gérard, and Muluh Godson. Explored how the OCaml ecosystem is managing the maintenance burden resulting from an increasing number of features available for editors by adopting the generic open standard Language Server Protocol (LSP). In his own words, Xavier said that “I presented some work carried out in the name of ‘ecumenism’ for OCaml editor support! How to make Merlin and OCaml-LSP-server interact to reduce the need for (heavy) maintenance between the wide variety of editors. We had to walk the line between minimising maintenance without losing functionality. The entire work is described in the paper: A New Era of OCaml Editing Powered by Merlin, Delivered via LSP”.

• Taming the Flat Float Array Optimisation: Tracking Separability in the Type System by Diana Kalinichenko and Richard A. Eisenberg. This talk presented an approach to flat float array optimisation that is used in OxCaml, which enabled previously rejected non-separable types to be used while maintaining compatibility with existing code and unlocking new optimisations for arrays of known non-float types.

Of course, there are more talks at the conference outside of the workshop, and this year, attendees were spoiled for choice with plenty of OCaml topics to explore!

Miscellaneous OCaml Talks

• A Guided Tour Through Oxidised OCaml by Gavin Gray, Anil Madhavapeddy, KC Sivaramakrishnan, Will Crichton, Shriram Krishnamurthi, Chris Casinghino, and Richard A. Eisenberg. This collaborative tutorial was a combined effort by members from Brown University, the University of Cambridge, Jane Street, IIT Madras, and Tarides. It took participants on a tour of the most significant extensions of OxCaml, Jane Street’s production compiler for performance-oriented programming, including fearless concurrency, data layout, and location control. The creators of the workshop encourage you to work through the slides online, try the activities, and give them feedback by filling in the quiz.

  In his blog post on the tutorial, Anil shared that “the tutorial itself [...] went fantastically! Both sessions were completely full, with participants online as well”.

• Functional Networking for Millions of Docker Desktops (Experience Report) by Anil Madhavapeddy, David J. Scott, Patrick Ferris, Ryan Gibb, and Thomas Gazagnaire from the University of Cambridge, Docker, and Tarides at the ‘Applications and SRC Talks’. This talk reflected on a decade of functional OCaml code in production as part of Docker’s container architecture across millions of desktops.

• Implicit Modules, a Middle Step Towards Modular Implicits by Samuel Vivien and Didier Rémy from Inria and PSL in the ML Family Workshop. Described a new proposal to extend OCaml with implicit module arguments, a long-term project with the first step, modular explicits, about to be integrated with mainstream OCaml.

  This was one of Xavier’s most memorable talks of the conference, and he described it as “A careful presentation that outlined the status of the project, what has been done, what remains to be done, and which is very promising for the future of OCaml!”

• The Wild West of Post-POSIX IO Interfaces by Anil Madhavapeddy from the University of Cambridge as a keynote speech in the Language Semantics & Type Systems track. The talk tracked the evolution of asynchronous and shared-memory I/O, from the POSIX programming model to OCaml 5’s Eio library.

  Anil distilled the most important part of his talk as follows: “So I made one key argument to the audience: it's time to accept that standards such as POSIX are now holding back the development of good language runtimes, and we need to embrace the diversity of highly concurrent, shared-memory interfaces.”

• Generating Hazel Programs From Ill-Typed OCaml Programs by Patrick Ferris and Anil Madhavapeddy from the University of Cambridge at the Type-Driven Development workshop. Highlighted the work on Hazel, a young programming language, and the development of a compiler to Hazel that can generate ill-typed code to help its development and testing.

• OCaml Blockly by Kenichi Asai of Ochanomizu University from the presentations for papers in the Journal of Functional Programming. This talk introduced OCaml Blockly, which is a block-based programming environment based on Google Blockly, but for OCaml. It is useful because it knows the scoping and type rules of OCaml: meaning that for any complete program in OCaml Blockly, its OCaml counterpart will compile successfully.

• Formal Semantics and Program Logics for a Fragment of OCaml by Remy Seassau, Irene Yoon, Jean-Marie Madiot, and François Pottier from Inria in the ICFP Papers track. Their talk shared work towards a formal definition of OCaml and a foundational program verification environment for the language by presenting a formal definition of a nontrivial sequential fragment of OCaml: OLang.

• A Tale of Two Lambdas: A Haskeller’s Journey Into OCaml by Richard A. Eisenberg from Jane Street at the Haskell track. Richard shared his experience with both the Haskell and OCaml ecosystems and communities, offering insights on how they are similar, how they are different and, most importantly, what they can learn from each other.

  This talk was a highlight for both Anil and David! Anil wrote in his blog post series that “This year, Richard Eisenberg was my absolute highlight with a superb session on what he's learnt from being the rare breed of someone steeped deeply both in Haskell and OCaml. The room was so packed for this talk that they had to create an overflow room streaming it in the corridors!” and David enjoyed the entire Haskell track, saying “Haskell did very nicely this year, with Richard Eisenberg’s likewise excellent keynote on the Friday”.

In addition to all the amazing OCaml talks, there were, of course, plenty of presentations on other topics and programming languages!

Honourable Mentions: Non-OCaml Talks

• Programming for the Planet. This was the second ever PROPL workshop after the first one was held in London last year. It hosted several talks centred around using functional programming methods to address challenges to biodiversity, climate, and the vast amounts of planetary data that the sector yields. Anil shared that “By far my favourite aspect of PROPL was the sheer ambition on display when it comes to leveraging the network effects around computer technology to accelerate the pace of environmental action”.

• Continuations in Music by Youyou Cong at ‘Continuations at Work’ explored the usefulness of continuations in computer programming languages and music. David thought it was “thought-provoking, and potentially (even) more to go further back in musical history to late mediaeval and early Tudor, where the pattern work of composition and imitative properties are even more abundant (continuations with combinators for inversion, canon, etc.)”.

• Polynomial-Time Program Equivalence for Machine Knitting by Nathan Hurtig, Jenny Han Lin, Thomas S. Price, Adriana Schulz, James McCann, and Gilbert Bernstein from the University of Washington, University of Utah, and Carnegie Mellon University. Presented their algorithm at the ‘Applications and SRC Talks’ to canonicalise algebraic representations of the topological semantics of machine-knitting programs. David commented that it stood out to him “not because I can follow the category theory, but because it was a beautiful exposition”.

• Join Points in Practice was a keynote by Simon Peyton Jones all about join points as a powerful optimisation tool. Both Xavier and David especially emphasised how enjoyable Simon’s presentation style is: “One should always attend a Simon P-J talk, and his packed Haskell keynote was no exception”, David said, and Xavier commented: “His presentation, his gift of the gab, and his enthusiasm make him the most impressive and enjoyable speaker I have ever seen. So seeing him live was a must for me, and I wasn't disappointed by the delivery”.

• Freer Arrows and Why You Need Them in Haskell by Grant VanDomelen, Gan Shen, Lindsey Kuper, and Yao Li at the Haskell track. This talk discussed freer arrows, an expressive structure that is amenable to static analysis. Xavier said: “I use arrows extensively for my website, and I thought it was great to see their defunctionalised (freer) representation with very clear examples!”

• Functional Art, Music, Modelling and Design (FARM) This workshop welcomed submissions across art, crafting, and design, and hosted talks on music, vector graphics drawing, and even coats of arms! My colleague Stephen Sherratt attended the workshop and presented the talk Software-defined Declarative Synthesiser Live-Coding in a Jupyter Notebook, where he generated music in real-time by writing Rust code into a Jupyter notebook. He also particularly enjoyed the talks Weft – Enabling Tidal on the Web by Matthew Kaney and William Payne, and Generalising Turtle Geometry: An Extensible Language for Vector Graphics Drawing by Alice Rixte.

ICFP 2026: See You Next Year!

As mentioned above, next year’s ICFP will take place in Indianapolis, Indiana, from August 24 to 29. It will be another great opportunity to meet developers, discuss everything functional programming and OCaml, and stay up to date with the latest changes and releases. Keep an eye on the website and ICFP’s socials to register, and, of course, submit your talks before the February deadline. We hope to see you there!

You can connect with us on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects. We look forward to hearing from you!

Bringing Emacs integration to OCaml’s LSP server benefits both the user and the maintainer. If you use Emacs and want to start using OCaml, or switch to a more simplified setup, check out the ocaml-eglot repository on GitHub to try the new Emacs minor mode.

This post will give you some background to the development of the new tool, as well as the benefits and limitations of LSP, and the features of ocaml-eglot. Let’s dive in!

The Problem: ‘Editor Burnout’

The goal of the ocaml-eglot project was to address a problem the engineers had dubbed editor burnout. Developers rely on editors to simplify their coding workflow, and over the years, the creation of more and more editor features has transformed editors into sophisticated, feature-rich development environments. However, all these features need to be added and maintained in every editor. Maintaining support for so many different features across different editors, including updating the support every time something changes on the language server's end, can quickly become untenable. ‘Editor burnout’ refers to the pressure this puts on maintainers.

In OCaml, the editor-agnostic server Merlin is used to provide IDE-like services. By providing contextual information about the code, Merlin lets developers use simple text editors to write OCaml and benefit from features that typically come from a fully integrated IDE. However, Merlin also had a high maintenance cost due to each code editor needing its own integration layer.

So, now that we understand the problem, what is the solution?

LSP and OCaml

LSP, or the Language Server Protocol, is a widely documented open protocol that standardises the interactions between an editor and a server providing IDE services. LSP defines a collection of standard features across programming languages, which has contributed to its widespread adoption. This adoption has made LSP a standard protocol across editors, including Visual Studio Code, Vim, Emacs, and many more.

The language server implementation for LSP in OCaml is ocaml-lsp. It uses Merlin as a library. It was originally designed to integrate with Visual Studio Code when paired with the vscode-ocaml-platform plugin. We can significantly reduce the maintenance burden by relying on LSP's defaults for editor compatibility and only providing support for OCaml-specific features. This benefits not only the maintainers, but also the user by ensuring the plugins remain performant, compatible, maintainable, and up-to-date.

LSP aims to be compatible with as many languages as possible, making some assumptions about how those languages are structured and function. Inevitably, these assumptions cannot cover all the features of every language. This is true of OCaml, where the editing experience relies on custom features outside the scope of the LSP.

The solution to this incompatibility is to create a client-side extension that covers what the editor’s standard LSP support does not. That way, we have both the basic LSP compatibility and an extension that adds support for OCaml-specific features. As we’ve hinted above, this has the added benefit of keeping the maintenance workload on the editor side down by delegating the standard LSP handling to the generic LSP plug-ins.

Some of these OCaml-specific editor features include type-enclosing, construct and navigation between holes, destruct, and search by types.

OCaml-Eglot

As an editor popular with the OCaml community, let’s take a brief look at how Emacs and OCaml work together. In Emacs, developers can attach a "buffer"/file to a major mode to handle a feature of a language like OCaml: features like syntax highlighting, for example. One file is always attached to just one major mode.

OCaml has four major modes:

• caml-mode: the original,
• tuareg: a full reimplementation of caml-mode and the most common choice by users,
• ocaml-ts-mode: an experimental version of caml-mode based on tree-sitter grammar,
• neocaml: an experimental full reimplementation of tuareg based on tree-sitter grammar.

Now, we can also attach one or multiple minor-modes to a file, and this is where ocaml-eglot comes into play. For example, we can use a major mode (we generally recommend Tuareg) and link ocaml-eglot to it as a minor mode, thereby attaching LSP features to all files in which Tuareg is active.

Eglot is the default LSP client bundled with Emacs, and ocaml-eglot provides full OCaml language support in Emacs as an alternative to Merlin integration. (By the way, thanks to the ocaml-eglot client using LSP’s defaults, its code size is a lot smaller than the traditional OCaml Emacs mode, which also makes it easier to maintain!).

The ideal user of ocaml-eglot is someone who is already an active Emacs user and wants to start using OCaml with minimal start-up hassle. The simplified configuration, automated setup, and consistency across different editors and languages are helpful both to people new to OCaml and to seasoned users with multiple editors, since they improve the workflow. The plugin supports all the features of the integration of Merlin into Emacs, merlin.el, meaning that users don’t lose any functionality with the new system. The ocaml-eglot project is also actively maintained, and users can expect regular future updates and a tool that evolves with the times.

Creating OCaml-Eglot

Let's peek behind the curtain at the development of ocaml-eglot. There are two common approaches that developers who implement server languages tend to use to add features outside of the LSP. These are Code Actions and Custom Requests:

• Code Action: A contextual action that can be triggered from a document perspective, can perform a file modification, and potentially broadcast a command that can be interpreted by the client. Code Actions are more ‘integrated’, which means that they sometimes even work ‘out of the box’ with the client. However, they are limited in terms of interactions and the command lifecycle.
• Custom Request: Not formally part of the protocol, but since LSP is a protocol layered on top of a regular server that can handle JSON RPC messages and responses, developers can still use arbitrary requests to provide extra features. Custom Requests give developers more power to add interactions and experiences, but always need specific editor integration.

The design process behind OCaml-eglot essentially boiled down to identifying all the features offered by merlin.el that were not covered by the LSP, and then adding them using Code Actions or Custom Requests. During this process, the developers asked themselves two questions to help them decide which approach to use:

• Should the feature be configured by arguments that are independent of the context: If the answer is yes, they used a Custom Request; if no, they used a Code Action.
• Does the feature require additional interaction such as choosing one option from a set of possible results?: If yes, they used a Custom Request; if no, they used a Code Action.

Of course, things were a little more complicated than this in reality, but it still gives you a good idea of the types of technical decisions the team made during development.

Try it Out!

Install ocaml-eglot by checking out its GitHub repository and following the instructions. When you have had a chance to test it out in your projects, please share your experience on OCaml Discuss to give other users an idea of what to expect and the maintainers an idea on what to improve!

Installing ocaml-eglot is just like installing a regular Emacs package. It is available on Melpa and can be installed in many different ways, for example with GNU’s use package. More detailed instructions are available in the repo’s readme, including instructions on recommended configurations for ocaml-eglot.

Features

Some of the features that ocaml-eglot comes with are:

• Error navigation: Quickly jump to the next or previous error(s).
• Type information: Display types under cursor with adjustable verbosity and navigate enclosing expressions.
• Code generation: Pattern match construction, case completion, and wildcard refinement via the ‘destruct’ feature.
• Navigation: Jump between language constructs like let, module, function, match, and navigate phrases and pattern cases.
• Search: Find definitions, declarations, and references. The team also recently introduced a new Xref Backend inspired by one used by Jane Street for years.

Check out the project's readme to discover the full list of commands offered by ocaml-eglot. The new mode is ‘agile’, meaning that the team can also incubate new features quickly, like the refactor extract at toplevel.

Until Next Time

For some really helpful background that goes into more detail than we did in this post, I recommend that you read the paper “A New Era of OCaml Editing Powered by Merlin, Delivered by LSP” by Xavier van de Woestyne, Sonja Heinze, Ulysse Gérard, and Muluh Godson.

You can connect with us on Bluesky, Mastodon, Threads, and LinkedIn or sign up to our mailing list to stay updated on our latest projects. We look forward to hearing from you!

This post will give you an overview of the release, including some background on Unikraft and performance graphs. You will already be familiar with most of this if you read our post on Discuss.

What is Unikraft and Why Did We Choose It?

Unikraft is a unikernel development kit: a large collection of components that can be combined in any configuration that the user wants in the unikernel tradition of modularity. Unikraft's scope is larger than that of Solo5 (a backend that MirageOS also supports). It aims to make it easy to turn any Unix server into an efficient unikernel.

In fact, the initial motivation behind exploring Unikraft as a MirageOS backend was to experiment and see what performance levels we could reach. We were particularly excited about using their Virtio-based network interface, as virtio is currently only implemented for one specific x86_64-only backend in Solo5. Some of the immediate performance differences we observed are detailed further down, but performance is not all we hope to gain from the Unikraft backend in the long run.

Unikraft is on the road to being multicore compatible (i.e., having one unikernel using multiple cores). While this is not ready yet and significant effort is needed to get there, it means that the MirageOS backend will eventually benefit from these efforts and support the full set of OCaml 5 features.

Furthermore, the Unikraft community (which is quite active) is experimenting with a variety of other targets, such as bare-metal for some platforms or new hypervisors (e.g. seL4). Any new target Unikraft supports can then be supported ‘for free’ by MirageOS too. For example, the Unikraft backend has already resulted in Firecracker being a new supported virtual machine monitor (VMM) for MirageOS.

Lastly, since Unikraft is POSIX-compatible (for a large subset of syscalls), it has the potential to enable MirageOS unikernels to embed OCaml libraries that have not been ported yet. This compatibility could be especially useful for large libraries which are hard to port (owl comes to mind).

How Does Unikraft Support Work?

Adding the new MirageOS backend required that we create or modify a series of components:

• An OCaml cross compiler that can build the new backend by building its corresponding runtime and providing a way to build unikernel images (instead of normal executables).
• New libraries for Unikraft system support, including its network and block devices.
• Support for the new backends in the mirage tool.

Using Unikraft with a QEMU or a Firecracker backend is as simple as choosing the unikraft-qemu or the unikraft-firecracker target when configuring a unikernel.

The OCaml/Unikraft Cross Compiler

To build the OCaml cross compiler with Unikraft, we used the Unikraft core, Unikraft lib-musl, and musl. The musl library is the C library recommended by Unikraft for building programs using the POSIX interface. The combination made it easy to build the OCaml 5 runtime, particularly because it provided an implementation of the pthread API, which is now used in many places in the runtime. It could also potentially make it easier to port some libraries that depend on Unix to work on Unikraft backends.

The OCaml cross compiler builds upon the work that has been upstreamed to ease the creation of cross compilers, using almost the same series of patches as for ocaml-solo5. So the only versions of the compiler that are currently supported for OCaml/Unikraft are OCaml 5.3 and 5.4. All the patches have been upstreamed to OCaml so there should no longer be any patches required by OCaml 5.5.

Note that we didn’t go with the full standard Unikraft POSIX stack, which includes lwIP to provide network support. We had a prototype at some point relying on lwIP to validate our progress on other building blocks, but it raised many incompatibility issues with the standard MirageOS network stack, so we dropped support for lwIP for now. Instead, we developed the libraries required to plug the MirageOS stacks into the low-level interfaces provided by the Unikraft core.

The New MirageOS Libraries for Unikraft Support

Unikraft support comes with packages using the standard names, mirage-block-unikraft and mirage-net-unikraft, to support the block and network devices. These libraries are implemented directly on top of the low-level Unikraft APIs, and therefore use virtio on both QEMU and Firecracker VMMs.

To evaluate the quality of the implementations for these devices, we ran a couple of small benchmarks using OCaml 5.3 and Unikraft 0.18.0. You can find the benchmarks (including the unikernels along with some scripts to set them up and run them) in the benchmarks directory in @Firobe’s fork of mirage-skeleton, benchmarks branch.

Network Device

To measure the performance of the network stack, we tweaked the simple network skeleton unikernel to compute statistics and used a variable number of clients all sending 512MB of null bytes. We have run this benchmark on both a couple of x86_64 laptops and on an LX2160 aarch64 board, all running a GNU/Linux OS.

We have observed a lot of variability in the performance of the solo5-spt unikernel (sometimes better, sometimes worse than unikraft-qemu) depending on the actual computer used, so these measurements should be taken with a grain of salt.

On two different x86_64 laptops:

On the LX2160 aarch64 board:

Block Device

To measure the performance of the block devices, we wrote a simple unikernel copying data from one disk to another. We can see that the performance of unikraft-qemu is lower than solo5-hvt for small buffer sizes, but fortunately, the situation improves with larger buffer sizes. We only ran this benchmark on an x86_64 laptop, as there is currently an issue with two block devices on aarch64 on Unikraft.

It is worth mentioning that I/Os can be parallelised, which also yields a significant performance boost. Indeed, mirage-block-unikraft can leverage the parallelised virtio backend of QEMU and Firecracker, which solves the problem of limiting I/Os to what the hardware supports in terms of both parallelism and sector size.

Current Limitations

1. In our tests, only Linux appeared to be well supported for compiling Unikraft at the moment. As a result, we have restricted our packages to that OS for now.
2. Unikraft supports various backends. At the moment, we’ve only added support and tested its two major ones: QEMU and Firecracker.

Try it Out

To try the new Unikraft backend for MirageOS, you need to use an OCaml 5.3 or 5.4 switch, so that you can install mirage and the OCaml/Unikraft cross compiler. The short version could be:

$ opam switch create unikraft-test 5.4.0 # or 5.3.0, and if needed
$ opam install mirage ocaml-unikraft-backend-qemu ocaml-unikraft-x86_64

See below for some explanations about the numerous OCaml/Unikraft packages. From then on, you can follow the standard procedure (see how to install MirageOS and how to build a hello-world unikernel) to build your unikernel with the Unikraft backend of your choice, which should boil down to something like:

$ mirage configure -t unikraft-qemu
$ make

Details About the Various Packages for the OCaml/Unikraft Cross Compiler

The OCaml cross compiler to Unikraft is split up into 14 packages (see the PR to opam-repository for more details) so that users can:

• Choose which of the backends (QEMU or Firecracker) and which of the architectures (x86_64 and arm64) they want to install, where all combinations can be installed at the same time.
• Choose which architecture is generated when they use the unikraft OCamlfind toolchain by installing one of the two ocaml-unikraft-default-<arch> packages.
• Install the ocaml-unikraft-option-debug to enable the (really verbose!) debugging messages.

Furthermore, virtual packages can be installed to make sure that one of the architecture-specific packages is indeed installed:

• ocaml-unikraft can be installed to make sure that there is indeed a unikraft OCamlfind toolchain installed.
• ocaml-unikraft-backend-qemu and ocaml-unikraft-backend-firecracker can be installed to make sure that the unikraft OCamlfind toolchain supports the corresponding backend.

Those virtual packages will be used by the mirage tool when the target is unikraft-qemu or unikraft-firecracker.

All those packages use one of two version numbers. The backend packages use the Unikraft version number (0.18.0 and 0.20.0 have been tested and packaged) while the latest OCaml cross-compiler packages use version 1.1.0.

Conclusion

We are still experimenting with this new backend. We expect to run it in production in the coming months, but it may need improvements nevertheless. Notably absent from this release is an early attempt to leverage Unikraft’s POSIX compatibility to implement Mirage interfaces instead of hooking directly to Unikraft’s internal components. This early version used Unikraft’s lwIP-based network stack instead of Mirage’s (fooling Mirage into thinking it was running on Unix), and it may be interesting to revisit this kind of deployment, in particular for easy inclusion of Unix-only OCaml libraries in unikernels.

We are eager for reviews, comments, and discussion on the implementation, design, and approach of this new Mirage backend, and hope it will be useful to others.

You can connect with us on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects. We look forward to hearing from you!

What is OCurrent?

OCurrent is both a small embedded domain-specific language (eDSL) for describing workflows and pipelines, and the wider CI infrastructure that powers much of the OCaml ecosystem. The OCurrent library enables developers to express build, test, and deployment logic directly in OCaml, with automatic dependency tracking and selective re-execution of steps when inputs change. This makes it particularly well-suited for long-running, continuously evolving systems where correctness and reproducibility are key. Pipelines written in OCurrent are self-adjusting: when a Git branch, Docker image, or external dependency is updated, the pipeline reacts automatically.

OCurrent underpins the continuous integration and build infrastructure for the OCaml community. The ocaml-ci service, which provides CI for OCaml projects hosted on GitHub, is implemented using OCurrent. Similarly, opam-repo-ci tests submissions to the opam repository. The same infrastructure is used for building and maintaining Docker base images and services like ocaml-docs-ci and ocaml-multicore-ci. Together, they keep the OCaml ecosystem’s packages, documentation, and base environments current across compiler versions, architectures, and operating systems.

How Will We Use the FLOSS Fund Grant?

This grant will enable us to dedicate time to maintaining OCurrent’s core components and the CI infrastructure, enhancing their performance and reliability, and refining the developer experience. The funding will help sustain the infrastructure that powers ocaml-ci and opam-repo-ci, as well as the automated build and deployment pipelines that many OCaml projects rely on.

For an ecosystem like OCaml’s, with its diversity of compiler versions, platforms, and tooling, having a reliable and type-safe pipeline engine is crucial. OCurrent enables reproducible builds and continuous integration, reducing friction for developers and ensuring that the ecosystem remains healthy and up to date. As described in our earlier post on the renovated ocaml-ci, OCurrent powers the “zero-configuration” CI experience that has become a cornerstone of OCaml’s development workflow.

A Note of Thanks

We’re grateful to FLOSS Fund for recognising the importance of this work and for supporting the continued development of the open-source infrastructure that keeps OCaml projects running smoothly. Thanks also to the many contributors, users, and maintainers of OCurrent, ocaml-ci, opam-repo-ci, and related projects. Your participation and feedback continue to shape the future of the OCaml ecosystem.

You can connect with us on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects. We look forward to hearing from you!

Let's dive in!

Immutable Arrays

Immutable arrays, as their name suggests, are like regular arrays in OCaml, with the major difference that their contents cannot be modified after they are created. Immutability is often a useful property for data structures and extending this to arrays provides various improvements over regular arrays.

Immutable arrays can improve safety in cases where mutation isn't required and only random-access packed memory is needed. In such a situation, an immutable array clearly communicates the design intention to the developer, improving their use of the code and improving safety as a consequence. Since the immutability property does not allow a function to change the contents, immutable arrays also improve the reasoning properties of the code, which verification tools can use. More concretely, a new predefined type 'a iarray and an Iarray module (with corresponding Iarraylabels) have been added to the Stdlib.

Lastly, immutable arrays can be safely coerced since there's no risk of inserting incompatible types. Consider this code example:

module Positive : sig
  type t = private int
  val make : int -> t
end = struct
  type t = int
  let make i = if i > 0 then i else invalid_arg "make"
end

let nums = [1; 2; 3; 4; 5; 6]

let lst = List.map Positive.make nums
let arr = Array.of_list lst
let iarr = Iarray.of_list lst

You can sum up lst with List.fold_left (+) 0 (lst :> int list), but if you do that with Array.fold_left (+) 0 (arr :> int array) you will be told Type Positive.t array is not a subtype of int array. Conversely, Iarray.fold_left (+) 0 (iarr :> int iarray) works, which is very useful, as the only workaround with an array is to coerce individually or - more often - copy the items to a new array, which is unfortunate (because the copy is just to satisfy type safety).

Pull Request #13097 introduced immutable arrays to OCaml 5. It is a feature Jane Street has been using internally in their OxCaml branch. The team at Tarides collaborated with Jane Street engineers to upstream this feature. The resulting PR sparked some great discussions in the community, and after considering their feedback, it was merged to form part of the 5.4 release.

Labelled Tuples

Labelled tuples allow the developer to label tuple elements, giving useful names to constructed values in cases where labelled function arguments allow them to name parameters. Reordered and partial patterns are resolved during type checking, and the labels are erased during translation to lambda. Labelled tuples do not support extracting an element of the tuple by the label.

One example of this being useful is when developers want to compute two values from a list without mixing them up. Labelled tuples can help prevent them from accidentally returning the pair in the wrong order or mixing up the order of the initial values. This code was given as a motivating example by the authors:

let sum_and_product ints =
  let init = ~sum:0, ~product:1 in
  List.fold_left ints ~init
    ~f:(fun (~sum, ~product) elem ->
          let sum = elem + sum in
          let product = elem * product in
          ~sum, ~product)

The function ~f has type sum:int * product:int -> int -> sum:int * product:int which enforces the ordering when constructing the return tuple.

Jane Street has been using labelled tuples internally for almost a year as part of their OxCaml branch, and reports that it has been a useful and popular feature. A core developer, David Allsopp, also benefited from it when he was working on the Relocatable OCaml Project. He used labelled tuples in the test harness for the feature.

To find out more, check out Chris's talk from the 2024 ML workshop, the ML workshop talk proposal, and PR #13498, which documents the changes.

Frame Pointers

Frame pointers are used to walk the stack of function calls in a program. Tools like profilers and debuggers use frame pointers to walk the stack and reconstruct the call graph for programs. Having this support means that third-party debugging and performance tools, like perf, eBPF, Dtrace, GDB, and LLDB now work much better with OCaml.

The history of frame pointers in OCaml is a long and somewhat complicated one. The compiler has had an option for frame pointers on AMD64 since OCaml 4.01, released in 2013. After the multicore update in OCaml 5, support returned for Linux/AMD64 (#11144) in the 5.1 release and macOS/AMD64(#13163) in the 5.3 release. In 5.4, support was added for ARM64 on both Linux and macOS, along with documentation on using frame pointers to profile OCaml code. The team has started work on supporting RISC-V, s390x, and Power architectures, which we hope to see implemented in the future.

For a fuller view of all the work, you can check out PRs #13500, #13575, #13635, and #13751, which have improved frame pointer support in OCaml 5.4. Tim McGilchrist also wrote a blog post on his website about implementing frame pointers in OCaml.

Atomic Field Accesses

Previously, OCaml 5 had limited support for atomic operations, only supporting them on the special 'a Atomic.t type. After a lot of discussion on the best solutions, consensus landed on introducing records with atomic fields to improve performance when implementing concurrent data structures. Instead of requiring a field to be of type 'a Atomic.t and introduce an indirection on that record field, having records with atomic fields is more efficient.

PR #13404 implements atomic operations, including two features described in 'atomic record fields' RFC. First, atomic record fields are now just record fields marked with an atomic attribute, and their reads and writes are compiled into atomic operations. Second, the PR implements atomic locations, a compiler-supported way to describe an atomic field within a record to perform atomic operations in addition to read and write operations.

Unloadable Runtime

OCaml 5.4 reintroduces 'memory cleanup upon exit' mode. The purpose of this mode is to enable something called the 'unloadable runtime'. Suppose you're running a program written in another programming language, and you use an OCaml library as a shared library. When control switches over from the OCaml library to the main program, you want the handover to happen cleanly, with all OCaml runtime resources being 'unloaded', including the stack, heap sections, code fragments, custom operations, buffers, and tables.

In the update from OCaml 4 to 5, support for the unloadable runtime was lost as multiple domains complicated matters. An OCaml program generally stops and exits once the main domain runs to completion, and the same behaviour needs to be adopted for multiple domains. When the main domain stops, the other domains are also terminated. Terminating all running domains was the trickiest aspect to implement since only stopping for garbage collection was originally supported. Now, with 'memory cleanup upon exit' mode, all domains can be terminated, and the runtime can be unloaded before control is handed back, ensuring a clean handover to a host program. Learn more in PR #12964, which introduced the feature in 5.4.

… And Many More!

GC Performance Improvements (Stephen Dolan, Nick Barnes, Gabriel Scherer, reviews by François Bobot, Josh Berdine, Damien Doligez, Tim McGilchrist, Guillaume Munch-Maccagnoni, benchmarking by Nicolás Ojeda Bär, and reports by Emilio Jesús Gallego Arias and Olivier Nicole)

This project improved the performance of garbage collection in the runtime. Changes to the way ephemerons are treated by the minor GC, allowing values from ephemeron keys to be collected and optimising them so they are not re-marked, have expanded their functionality whilst keeping the minor GC performant. A change to the major GC’s pacing, fixing a bug where the work_counter would get out of sync with the alloc_counter, also boosted performance. Finally, a new Gc.ramp_up callback will be introduced, allowing users to mark ramp-up phases of memory consumption and preventing slowdowns as a result of collection work being done twice. The work is spread across several PRs, including #13643, #13827, #13736, #13300, and #13861.

Software Prefetching Support (Tim McGilchrist, review by Nick Barnes, Antonin Décimo, Stephen Dolan and Miod Vallat)

5.4 enables software prefetching instructions for several architectures, including ARM64, s390x, PPC64, and RISC-V. This feature advises the processor to 'pre-fetch' data from slower memory and store it in faster memory before it is needed. This can help improve the overall performance of programs, and in fact, the PR #13582 contains several benchmarks showing speed-ups as a result of the changes.

Review of Locking in the Multicore Runtime (Guillaume Munch-Maccagnoni, review by Gabriel Scherer, tests by Jan Midtgaard)

Part of the task between updates is to identify and address problems. In PRs #13227 and #13714, issues surrounding the caml_plat_lock_non_blocking and caml_plat_lock_blocking caused deadlocks in the runtime. After auditing and testing the code to see what was going wrong, tighter constraints about when and how the commands could be mixed helped solve the deadlocking.

A Few Bug Fixes

#13605 (Samuel Vivien, review by Florian Angeletti, Richard Eisenberg and Jacques Garrigue)

This PR adds a check to detect and prevent errors occurring as a result of generating typing constraints. When users define a parametrised type and create a constraint by binding that type with an as, there were times when a constraint would not behave as expected. By adding an error, this PR prevents users from having constraints and introduces unwanted behaviour.

#13812 (Samuel Vivien, review by Gabriel Scherer)

This PR adds another check that tests the validity of the type variable name on the right-hand side of _as_. The added test reduces friction and confusion for the user.

#13895 and #13691 (Jan Midtgaard, review by Miod Vallat, Sadiq Jaffer and Antonin Décimo)

Gc.control had four underlying globals that would cause data races when tested with TSan. Jan tested them and rewrote them to be atomic, which resolved the racing and brought them in line with other globals which were already atomic.

What’s Next?

Work continues! Performance improvements to the garbage collector that aim to reduce the performance gap between versions 4.14 and 5.x include GC pacing and mark delay work. Relocatable OCaml is another ongoing project, whose RFC was accepted in principle in March. It wasn't quite ready for 5.4, but will hopefully finally be arriving in OCaml 5.5!

You can connect with us on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects. We look forward to hearing from you!

The Start of Ocsigen

So, how did the idea of a web framework for OCaml come about? Ocsigen began as a research project under the Research Institute for Foundations of Computer Science (IRIF), aiming to introduce support alongside the requisite tools for web development in OCaml. At the time of its inception (2005), there were no mature web frameworks for the language, and the founders Vincent Balat and Jérôme Vouillon wanted to implement a full framework that would support professional-level projects at scale.

The reasoning behind choosing OCaml for this project was to take advantage of the language’s expressive type system. This system can check the properties of programs at compile time, reduce development time, make it easier to refactor code and add new features, and ensure that existing features are working correctly. The research paper titled Ocsigen: Typing Web Interaction With Objective Caml, written by Vincent and based on the broader Ocsigen project, made two claims on which the design of Ocsigen was founded: firstly, that web languages could (and should) take greater advantage of static typing, and second, that functional programming is a good fit for web programming.

Continuations, JavaScript, and Multi-Tier Programming

The implications of their first claim meant that Vincent and Jérôme focussed on ensuring the correctness of generated pages using static typing – being able to statically check that links and forms match dynamic pages operating as typed services.

The second claim was inspired by several articles positing that web programming would benefit from taking advantage of continuations. Continuation is a concept in programming that refers to an abstract representation of the control state of a computer program. Functional programming can use and manipulate continuations in a straightforward and easy way. In web programming, continuations are a very elegant solution to what is known as “the back button problem”. Vincent noticed that, at the time, few tools took advantage of this opportunity and determined to implement continuations along with other functional programming features in Ocsigen. As a result, the framework has added whole new functionalities to aspects of traditional web programming using continuations including typed forms, typed links, and advanced session handling using scoped references.

Furthermore, the Ocsigen team implemented Js_of_ocaml, one of the first compilers to JavaScript, enabling developers to use OCaml both on the server and client side.

Finally, the Ocsigen team invented multi-tier programming. In multi-tier programming, both ‘sides’ of an application are written as a single program. This makes communication between the server and client straightforward, allowing the user to generate web pages from the server or the client side (with the application being indifferent to which side you choose). For the first time, this allowed developers to mix app development and traditional website development in the same app! It also opened up the possibility for users to write mobile applications with the exact same code as they wrote web applications.

What is Ocsigen?

Ocsigen consists of several independent open-source projects, all available on GitHub. In this post, we’ll introduce you to the most used ones, but if you want to discover more, check out Ocsigen's website.

• Lwt: is a concurrency library that handles I/O operations using promises. Promises are references that will be filled asynchronously. One of the biggest advantages when calling a function that returns a promise is that it will not require a new stack or process. This ensures a high-speed, efficient call. Recently, Ocsigen is moving to using Eio as a concurrency library.
• TyXML: is a library used to build statically correct HTML and SVG documents. It functions largely like HTML with a combinator implementing HTML attributes and elements, so it’s easy to pick up and use for most programmers. However, with TyXML, you can use OCaml to manipulate elements, and invalid markup text will yield a type error, which helps you write cleaner code. There are standalone examples available in the TyXML GitHub repo to help you get started.
• Js_of_ocaml: is a compiler for converting OCaml bytecode to JavaScript, allowing users to run pure OCaml programs in JavaScript environments. Js_of_ocaml is easy to install, works out-of-the-box with a lot of existing bindings to browser APIs, and generates performant programs. In addition, the Js_of_ocaml repo now also contains Wasm_of_ocaml, another compiler (originally a fork of the former) that compiles to WebAssembly targets.
• Eliom: is a framework that enables users to implement multi-platform applications on web browsers and mobile devices in a modern programming style. It is not a new language but rather an extension of OCaml. Its main features are that it provides high-level expressive concepts that enable developers to program complex behaviours in very few lines of code; supports higher security by implementing the OCaml type system and checking properties of applications at compile time; and makes it possible to write client-server applications as single programs using a multi-tier extension of OCaml.
• Ocsigen Server: is a web server, as the name suggests. It is available as an executable and as a library and supports a wide variety of services, including static files, redirection, reverse proxy, CORS, page compression, authentication, and more.
• Ocsigen Toolkit: is a collection of resources like widgets and other utilities that users may want for their web applications. Usefully, most of the widgets are compatible with mobile programming thanks to their ability to be produced on the server side of the client using the same code.
• Ocsigen Start: is an application template containing many standard features, including user management, notifications, and code examples. It can be used to learn the Ocsigen framework or to quickly create a prototype. There is a demo online that you can try before installing the application, and it’s also available on mobile.

A quick note on Eio: Eio is an effects-based direct-style I/O library compatible with OCaml 5 including multicore. Direct-style concurrency enables the user to write code in a natural style without taking into consideration which code is concurrent and which isn’t (thereby eliminating the well-known function colouring problem). Eio emphasises performance, making the most of new kernel I/O interfaces for enhanced parallelism efficiency and security, with parts of it being formally verified and using low-level and high-level interfaces. You can read more in one of our blog posts on Eio.

BeSport

The most prominent user of Ocsigen is BeSport. BeSport is a social networking platform created for sports clubs, amateurs, and pros. Teams and clubs can share content with their fans, and users can follow results, news, and statistics for their favourite professional teams or their own amateur leagues.

Choosing Ocsigen enabled the team behind BeSport to quickly build a feature-complete social network, functioning both as a web application and as Android and iOS applications. They achieved this under the constraints of a start-up: limited funding, unclear and rapidly evolving specifications, and a need to build clean and sustainable code. Their experience encapsulates the benefits of the Ocsigen programming model, which fosters fast and reliable development.

Until Next Time

You can connect with us on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects. We look forward to hearing from you!

Kick off was on the 8th of September. For the next 3 months, Parsimoni will benefit from Techstars’ mentorship, network, and investment to drive their mission of making satellite-based resources accessible to all.

Parsimoni’s SpaceOS

Parsimoni is developing SpaceOS, a next-generation operating system tailored to satellites and their payloads. By using unikernel technology (see MirageOS for another example), the size of each application is optimised to its minimal viable state, reducing the attack size and improving performance. The resulting efficiency is valuable for users – reducing costs, extending mission capabilities, and allowing for more advanced processing in orbit – and also better for the planet. More efficient, multi-purpose satellites make better use of limited resources and create new possibilities for the satellite industry.

These advantages, along with its security-by-design approach (including PQC) and high adaptability to new features and applications, make SpaceOS the perfect candidate for use cases like AI-powered earth observation, secure mission operations, and next-generation space marketplaces.

Parsimoni uses OCaml to build their software, benefitting from the language's security guarantees, and this approach is garnering attention from a world-leading accelerator. It illustrates how Tarides' mission of building mission-critical systems in OCaml is a recipe for success – in the FinTech sector, space sector, and beyond! Visit Parsimoni’s website to learn more about SpaceOS and its future development and deployment.

Techstars

Techstars is an accelerator with a global network that has been helping founders launch and grow their companies since 2006. Their three-month programme is designed to help startups put together all the pieces they need for success. This includes assigning mentors to each company, fostering business storytelling, providing fundraising opportunities and guidance on strategy, workshops, and networking. Techstars alumni include the very successful Chainanalysis, Zipline, DataRobot, and Alloy.

For their autumn 2025 cohort, Techstars have selected startups with a strong focus on innovation and meeting future markets. Parsimoni is in the space accelerator group, and other groups include healthcare, the future of food, and the future of finance. Over 50 of the startups are also using AI in their offers.

What’s Next

We’re looking forward to seeing how Parsimoni will rise to the challenge over the next few months. If you want to keep up with them and SpaceOS, follow them on LinkedIn and keep an eye out for updates! Stay tuned to our blog – we will keep you updated about how Parsimoni is enabling the next revolution of space-based innovation.

You can connect with us on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects. We look forward to hearing from you!

What is Ortac?

Ortac aims to give OCaml programmers easy access to dynamic formal verification, also known as specification-driven testing. At its core, it translates computable Gospel terms into equivalent OCaml expressions. Different Ortac modes can then use these translations to generate OCaml code. This post is about the Ortac/QCheck-STM mode that generates black-box model-based state-machine tests based on the QCheck-STM framework.

Ortac/QCheck-STM Mode

QCheck is a property-based testing framework inspired by QuickCheck. As the name implies, the idea behind property-based testing is to check a property about a function, generally expressed as an equation involving the inputs and outputs of the function, against randomly generated inputs.

In contrast, QCheck-STM checks the behaviour of randomly generated sequences of function calls against a model. The model is implemented as a state machine (hence the STM). Testing a sequence of function calls helps users discover more bugs, especially when a mutable state is involved. In order to use QCheck-STM on a library, the users have to specify which type is the center of attention, also called System Under Test, or SUT for short. They also have to provide a functional model for the SUT, equipped with a next_state function computing the new model given a function call. You can read more about QCheck-STM in a previous post on our blog and in a paper on parallel testing libraries for OCaml 5.

One of the positives of using Ortac/QCheck-STM is that with just some Gospel annotations, a Dune rule, and a configuration file, we can benefit from QCheck-STM tests. Another bonus is that in case of failure, the generated tests will provide a bug report containing the piece of Gospel specification that has been violated, a runnable scenario with the actual returned values to reproduce the failure and, if available, the expected returned value of the failing command according to the function specification.

Since the previous post, the Ortac tool has been improved in several ways. Version 0.4.0 brought support for keeping track of multiple Systems Under Test in the generated tests, all thanks to Nikolaus Huber's work. Then, version 0.5.0 brought support for testing higher-order functions, thanks to Jan Midtgaard. Finally, version 0.6.0 improved the computation of the expected returned value based on the specifications.

The reader curious about how Ortac/QCheck-STM works internally can refer to this paper. The rest of this post will adopt a more practical perspective, using the priority queue as a running example.

Project Setup

Let's first explore a project setup, how to write the Gospel specification of the API, and finally, how to integrate Ortac/QCheck-STM with Dune.

Here is the project's structure:

.
├── dune-project
├── priority-queue.opam
├── src
│   ├── dune
│   ├── priority_queue.ml
│   └── priority_queue.mli
└── test
    ├── dune
    ├── dune.inc
    ├── priority_queue_config.ml
    └── priority_queue_tests.ml

3 directories, 9 files

The idea behind Ortac/QCheck-STM is to not write the tests. Compared to a more traditional project, priority_queue.mli contains some Gospel specifications describing the expected behaviour of the functions, and the files dune.inc and priority_queue_tests.ml are generated by Ortac.

The priority_queue_tests.ml contains the generated QCheck-STM tests for the Priority_queue module, based on the Gospel specifications contained in the priority_queue.mli file and the priority_queue_config.ml file provided by the user. Furthermore, dune.inc contains the generated Dune rules to generate priority_queue_tests.ml and attach the execution of these tests to the runtest alias.

Generating dune.inc is the job of the Ortac/Dune mode, which is called by a hand-written dune rule in the dune file.

Due to how Dune's promote mode interacts with the include stanza, we must create an empty dune.inc. Another possibility is to use a dynamic_include rule (see the rule generation chapter in the Dune docs for more details).

Generating priority_queue_tests.ml is the job of the Ortac/QCheck-STM mode, which is called by a generated dune rule in dune.inc.

Ortac/QCheck-STM is provided by the ortac-qcheck-stm package, and Ortac/Dune by the ortac-dune one. We need to declare these two packages as dependency for our project.

Writing Some Gospel Specifications

Let's take a look at the interface file for our priority queue, where the Gospel specifications are stored.

We begin by declaring the OCaml abstract type of a priority queue alongside its logical specification:

(*@ open Sequence *)

(*@ type 'a priority = 'a * integer *)

(*@ type 'a priority_queue = 'a priority sequence *)

type 'a t
(*@ mutable model contents : 'a priority_queue
    with t
    invariant let q = t.contents in
              forall i.
              1 <= i < length q
              -> snd q[i-1] >= snd q[i] *)

Gospel annotations are written inside special comments opened with (*@. We can open modules from the logical library that Gospel provides. The Sequence Module contains the definition of a mathematical sequence and some operations. Ortac comes with an implementation of the Gospel logical library. We can also declare Gospel types using the same syntax as the OCaml one.

Gospel annotations immediately following an OCaml type or val are attached to it, a bit like documentation comments. So, in the example above, mutable model contents ... is the specification for the type 'a t. Gospel is a specification language based on models. Thus, in the first line of the specification, we give our OCaml type a model named contents. The model is also marked as mutable. That doesn't mean that the model itself is mutable but that the model of an 'a t may change when that 'a t is mutated. It is a way of specifying that the OCaml type 'a t is mutable.

Now, our model contents has a Gospel type: 'a priority_queue. If we unfold the definitions of the Gospel types priority_queue and priority given just before in the file, this means that we will see OCaml values of type 'a t as mathematical sequences of pairs of an element and an integer representing this element priority. There are other models we could have reasonably chosen, but let's go with this one. Let's also note that the model we choose is not directly related to the actual implementation. The logical model should make sense for any possible implementation.

We also use the invariant mechanism from Gospel to express that we keep the elements in decreasing order of priority in the model. This is not necessary, but it makes expressing the inspection and the evolution of the logical model a lot easier.

Now, we need to be able to express three fundamental operations precisely in our model: inserting a new element with its associated priority, looking at the next element, and deleting the next element.

(*@ function insert (q : 'a priority_queue)
                    (a : 'a)
                    (i : integer)
                    : 'a priority_queue =
      let higher = filter (fun x -> snd x > i) q in
      let equal = filter (fun x -> snd x = i) q in
      let lesser = filter (fun x -> snd x < i) q in
      higher ++ snoc equal (a, i) ++ lesser *)

(*@ function peek (q : 'a priority_queue) : 'a option =
      if q = empty
      then None
      else Some (fst (hd q)) *)

(*@ function delete (q : 'a priority_queue) : 'a priority_queue =
      if q = empty
      then q
      else tl q *)

The insert function does all the specification heavy lifting and is responsible for maintaining the invariant we've declared for our logical model. Note that Ortac/QCheck-STM will include invariant verifications in the generated tests!

The insert function is also the place where we choose to store the elements of the same priority in a FIFO manner by using the function snoc from the Gospel logical library.

The way insert is written makes peeking and deleting straightforward. In both cases, it suffices to look at and delete the first element of the sequence.

As we can see, this is similar to a very naive trusted functional implementation of the logical model. This makes a lot of sense if we think about it! As Ortac compiles Gospel specifications into OCaml code, it consumes computable specifications. This process also looks like part of what we would have written if we had been using QCheck-STM directly. These three functions are the ones necessary to implement the functional state-machine that the QCheck-STM tests will rely on, whether hand-written or Ortac-generated. One of the benefits of using Ortac/QCheck-STM is that it will test the Gospel implementation of these functions against the model's invariants for free!

We could argue that using an ordered list as a model for a priority queue is inefficient. But, as we are in a testing situation, performance matters less than the correctness of the model.

We now have all the necessary vocabulary to talk about the behaviour we expect from a priority queue:

val empty : unit -> 'a t
(*@ q = empty ()
    ensures q.contents = Sequence.empty *)

val insert : 'a t -> 'a -> int -> unit
(*@ insert q a i
    modifies q.contents
    ensures q.contents = insert (old q.contents) a i *)

val peek : 'a t -> 'a option
(*@ o = peek q
    ensures o = peek q.contents *)

val extract : 'a t -> 'a option
(*@ o = extract q
    modifies q.contents
    ensures o = peek (old q.contents)
    ensures q.contents = delete (old q.contents) *)

Gospel function's specification takes the form of a contract. The first line of which is a header naming the arguments and the returned value. What follows is a sequence of clauses describing the expected behaviour. Note that in these clauses, the names insert, peek, and delete refer to the Gospel functions defined above.

Here, the functions' contract is pretty straightforward. It is worth noting that in order to be able to include a function in the tests, Ortac/QCheck-STM asks that whenever a new SUT is created, or an existing one is modified, the contract contains a post-condition (an ensures clause) describing, again, in a computable way, the value of the related model.

When there are other post-conditions, they are added to the tests. Besides, when a post-condition is describing, in a computable way, the output value, Ortac/QCheck-STM will use it to include information about the expected returned value in the bug report in case of test failure.

Integrating Specification-Driven Tests With a Dune Workflow

Now, with a bit more effort, specification-driven testing is just a dune runtest away!

The first thing Ortac/QCheck-STM needs is a minimal configuration file in order to know how to build the QCheck-STM test suite we want. The minimal configuration consists of defining the sut type and the init_sut value.

The sut type is the type we want to focus the tests on, and the init_sut value is how to create an initial value to start the tests.

The configuration file also contains some additional information. Here we shadow the QCheck generators for ints and chars so that we only deal with three levels of priorities and readable elements in the tests.

type sut = char t

let init_sut = empty ()

module Gen = struct
  let int = oneofl [0;1;2]
  let char = char_range 'a' 'z'
end

Finally, we want to be able to generate and launch the tests with a dune runtest. Thanks to the Ortac/Dune plugin, we only need to write one rule:

(rule
 (alias runtest)
 (mode promote)
 (deps
  (:specs %{project_root}/src/priority_queue.mli))
 (action
  (with-stdout-to
   dune.inc
   (run ortac dune qcheck-stm %{specs}))))

(include dune.inc)

Other than that, you are all set to implement a priority queue against specification-driven generated tests!

Current and Future Work

As mentioned in the repo, Ortac/QCheck-STM has already proven itself useful by discovering and helping fix a number of issues.

Thanks to Charlène Gros, the Ortac/Wrapper mode has just been released. Ortac/Wrapper consumes Gospel annotations to generate runtime assertion checking. Given an annotated OCaml interface file, this plugin will generate a new module with the same signature but with an implementation instrumented with assertions taken from the Gospel specifications. Each function is wrapped with an assertion of its pre- and post-conditions.

Another project is to make Ortac/QCheck-STM also target QCheck-STM/Domains. For now, Ortac/QCheck-STM only generates QCheck-STM tests for testing the library in a sequential context. One of the strengths of QCheck-STM is that it provides a way to test mutable data structures in a parallel context using domains.

The whole project is under active development and should continue to evolve. We are also committed to open-source software, so if you want to take Ortac for a spin (and I encourage you to), please don't hesitate to ask questions, contribute issues, or even PRs!

Until Next Time

You can connect with us on Bluesky, Mastodon, Threads, and LinkedIn or sign up to our mailing list to stay updated on our latest projects. We look forward to hearing from you!

As part of my Tarides internship (on the editor side), I specified several useful commands, inspired by competitors and materialised in the form of RFCs, subject to discussion. There were multiple candidates, but we found that expression extraction to toplevel was the most suitable for a first experiment. Since it touched on several parts of the protocol and required tools that could be reused for other features, it was important to design the system with extensibility and modularity in mind.

In this article, I will present the results of this experiment, including the new command and some interesting use cases.

Examples

Expression extraction to toplevel will select the most inclusive expression that fits in your selection and propose to extract it. In this case, extract means that the selected expression will be moved into its own freshly generated let binding top level.

Extracting Constants

Here is a first example: Let's try to extract a constant. Let’s assume that the float 3.14159 is selected in the following code snippet:

let circle_area radius = 3.14159 *. (radius ** 2.)
                      (* ^^^^^^^ *)

The extract action code will then be proposed, and if you apply it, the code will look like this:

let const_name1 = 3.14159
let circle_area radius = const_name1 *. (radius ** 2.)

Here is an illustrated example (based on an experimental branch of ocaml-eglot):

We can see that the expression has been effectively extracted and replaced by a reference to the fresh let binding. We can also observe that in the absence of a specified name, the generated binding will be named with a generic name that is not taken in the destination scope. You also have the ability to supply the name you want for extraction.

For example, here is the same example where the user can enter a name:

But the refactoring capabilities go much further than constant extraction!

Extracting an Expression

In our previous example, we could speculate about the purity of the expression, since we were only extracting a literal value. However, OCaml is an impure language, so extracting an expression into a constant can lead to unintended behavior. For example, let's imagine the following snippet:

let () = 
  let () = 
    print_endline "Hello World!";
    print_endline "Done"
  in ()

In this example, extracting into a constant would cause problems! Indeed, we would be changing the semantics of our program by executing both print statements beforehand. Fortunately, the command analyses the expression as not being a constant and delays its execution using a thunk — a function of type unit -> ....

As we can see, our goal was to maximise the production of valid code, as much as possible, by carefully analysing how to perform the extraction. This is all the more challenging in OCaml, which allows for arbitrary (and potentially infinite) nesting of expressions.

Extracting an Expression That Uses Variables

The final point we’ll briefly cover is the most fun. Indeed, it’s possible that the expression we want to extract depends on values defined in the current scope. For example:

let z = 45

let a_complicated_function x y = 
  let a = 10 in 
  let b = 11 in 
  let c = 12 in 
  a + b + c + (c * x * y) + z

In this example, the extraction of the expression a + b + c (c * x * y) + z will be placed between z and a_complicated_function. As a result, z will still be accessible; however, x, y, a, b, and c will be free variables in the extracted expression. Therefore, we generate a function that takes these free variables as arguments:

Identifying free variables was one of the motivations for starting with this command. We are fairly certain that this is a function that we will need to reuse in many contexts! Note that the command behaves correctly in the presence of objects and modules.

A Real World Example

Let’s try to extract something a little more complicated now. Let’s assume we have the following code and we want to refactor it, for example, by extracting the markup type pretty print logic outside our pp function.

type t = markup list
and markup = Text of string | Bold of string

let show doc =
  let buf = Buffer.create 101 in
  let bold_tag = "**" in
  List.iter
    (fun markup ->
      Buffer.add_string buf
      @@
      match markup with
      | Text txt -> txt
      | Bold txt -> bold_tag ^ txt ^ bold_tag)
    doc;
  Buffer.contents buf

We can observe that bounded variables in the extracted region are now passed as arguments, and the extracted function is properly replaced by a call to the new show_markup generated function.

let show_markup buf bold_tag =
 fun markup ->
  (Buffer.add_string buf)
    (match markup with
    | Text txt -> txt
    | Bold txt -> bold_tag ^ txt ^ bold_tag)
let show doc =
  let buf = Buffer.create 101 in
  let bold_tag = "**" in
  List.iter (show_markup buf bold_tag) doc;
  Buffer.contents buf

Here is an example of how it is used. Impressive, isn't it?

Editor Support

To understand how this new Merlin command can be properly used in your favourite editor, we have to take a closer look at the functioning of the Language Server Protocol. The LSP supports two mechanisms to extend the existing protocol with new features. First, there is code action, which allows us to perform multiple LSP commands sequentially. This kind of request has the merit of working out of the box without requiring any plugin or specific command support on the editor side (which oils the wheels for maintenance). Secondly, there are custom requests, which are more powerful than code actions and enable custom interactivity. So, if you want to prompt the user, a custom request is the way to go. The price you have to pay for this power is to have client-side support implemented for each custom request in every editor plugin.

The current editor team approach is as follows: For each of Merlin's commands that don't map directly to a standard LSP request, we provide a code action associated with the Merlin command and potentially a dedicated custom request if the feature requires custom interactivity. Regarding the ‘extract’ feature, the associated code action does not allow us to choose the name of the generated let binding, but the custom request does.

What’s Next?

I hope this new command helps you get even more productive in OCaml! Don’t hesitate to experiment with it and report any bugs you encounter.

The development of Merlin’s refactoring tools was part of a broader vision to improve OCaml editor support and perhaps claim an editor experience similar to JetBrains IDE in the future!

The work done on the extract command gives us the opportunity to identify various problems pertaining to refactoring (substitution, code generation) and potentially to make the connection to refactoring commands that already exist in Merlin (like open refactoring and project-wide renaming). The next step is to add a small toolbox library in Merlin dedicated to refactoring in order to develop even more refactor actions. I hope this is just the first refactoring feature of a long series.

If you're curious and want to take a look at the feature, it's split into several PRs:

• ocaml/merlin#1948 which implements the extraction logic on the Merlin side and exposes it in the protocol,
• ocaml/ocaml-lsp#1545 which exposes the Custom Request enabling the use of the LSP-side functionality,
• ocaml/ocaml-lsp#1546 which exposes an Action Code that allows the functionality to be invoked without additional formalities on the Editor side,
• tarides/ocaml-eglot#65 which implements extraction behaviour in OCaml-Eglot, invocable either from a type enclosing or directly as a classic Emacs command.

All of these PRs are currently under review, and should be merged soon!

A big thanks to Xavier, Ulysse, and all the people that helped me during this internship. It was pretty interesting!

You can connect with Tarides on Bluesky, Mastodon, Threads, and LinkedIn or sign up to our mailing list to stay updated on our latest projects. We look forward to hearing from you!

Why do We Care About OCaml.org?

Tarides supports the maintenance and development of OCaml.org, OCaml’s home on the web. Our engineers have spent significant time collaborating with all corners of the OCaml community to update the website, including improving the design, accessibility, documentation, and much more. We continue to fund projects that implement new features that the OCaml community wants and maintain others that they have come to rely on.

As an open-source, collaborative, and shared resource for the ecosystem, OCaml.org is truly a public common. The OCaml Cookbooks are just one example of its many features, which also include tutorials, documentation, news, and an OCaml playground. We encourage more contributors, from sponsors to maintainers, to join the many others in supporting this important resource.

What is the OCaml Cookbook?

The OCaml Cookbook is a collection of ‘recipes’, instructions on how to complete tasks as part of projects using open source libraries and tools. The same task can have multiple recipes, each with its unique combination of resources. The result is a varied collection of projects that help users adopt new techniques, try new tools, and gain confidence in OCaml. OCaml’s book currently has recipes on compression, single-threaded concurrency, cryptography, and more!

OCaml is far from the only language to have a cookbook, and the team was inspired to create one by the popular Rust Cookbook, as well as Go’s Go by Example introduction to the language.

How to Contribute

The team behind the cookbook are always looking for new contributions, and creating a new recipe is straightforward if you follow the contributing instructions.

To add a new recipe to the cookbook, you will need to start by finding the OCaml.org repo on GitHub. To add a recipe to an existing task, find the task in the data/cookbook/tasks.yml section, go to the task’s folder inside data/cookbook/ which will have the same name as the task’s slug, and create an .ml file with the recipe and a YAML header with metadata about the recipe.

If the recipe you want to add doesn’t match an existing task, you will need to create a new task first. To add a task you will need to make an entry in the data/cookbook/tasks.yml file. When adding a new task in this file, the title, description, and slug are mandatory fields to fill in, and the task has to be located under a relevant category. You can even create new categories to organise entire groups of new tasks should you wish to do so.

Submitting a recipe will create a pull request, which the group of cookbook moderators will review and, if approved, merge into the website. When picking a recipe to contribute, you should bear the general guidelines in mind: choose a task that you think is relevant to a wide audience; write correct, clear, code that compiles without errors; and check that the packages you’ve chosen and the code are ready for use in production. That’s it, you’re ready to publish!

What Does a Recipe Look Like?

Let’s take a quick look at a recipe in action. The Salt and Hash a Password with Argon2 recipe in the Cryptography section shows you how to use the opam package argon2 to configure password hashing based on OWASP recommendations and Argon2 defaults. Be sure to check out the recipe on OCaml.org for the full context and nice formatting!

The recipe includes the code snippets for the configuration:

let t_cost = 2 and
    m_cost = 65536 and
    parallelism = 1 and
    hash_len = 32 and
    salt_len = 10

The hash output length:

let encoded_len =
  Argon2.encoded_len ~t_cost ~m_cost ~parallelism ~salt_len ~hash_len ~kind:ID

Generating a salt string:

let gen_salt len =
  let rand_char _ = 65 + (Random.int 26) |> char_of_int in
  String.init len rand_char

Returning an encoded hash string for the given password:

let hash_password passwd =
  Result.map Argon2.ID.encoded_to_string
    (Argon2.ID.hash_encoded
        ~t_cost ~m_cost ~parallelism ~hash_len ~encoded_len
        ~pwd:passwd ~salt:(gen_salt salt_len))

And finally, verifying if the encoded hash string matches the given password:

let verify encoded_hash pwd =
  match Argon2.verify ~encoded:encoded_hash ~pwd ~kind:ID with
  | Ok true_or_false -> true_or_false
  | Error VERIFY_MISMATCH -> false
  | Error e -> raise (Failure (Argon2.ErrorCodes.message e))

let () =
  let hashed_pwd = Result.get_ok (hash_password "my insecure password") in
  Printf.printf "Hashed password: %s\n" hashed_pwd;
  let fst_attempt = "my secure password" in
  Printf.printf "'%s' is correct? %B\n" fst_attempt (verify hashed_pwd fst_attempt);
  let snd_attempt = "my insecure password" in
  Printf.printf "'%s' is correct? %B\n" snd_attempt (verify hashed_pwd snd_attempt)

Contribute to the CookBook!

We invite you to take a look at the existing recipes up on the website and bring your own contributions to the book. If you have questions or want input on a recipe, OCaml’s Discuss forum is a great place to post to get tips and feedback.

Stay in touch with Tarides on Bluesky, Mastodon, Threads, and LinkedIn. We look forward to hearing from you!

We are always excited about projects that bring new features to OCaml and improve it for its users. This post will give you an overview of the different features being developed under the OxCaml umbrella and how Tarides is collaborating with Jane Street on the project.

What is OxCaml

OxCaml is an open-source branch of OCaml that incorporates several extensions designed to help write multicore and high-performance OCaml code.

Jane Street relies on OCaml’s sweet spot. OCaml lets you write code quickly, code that is performant, code that solves complex problems, and code that can be trusted, thanks to its strong focus on correctness-by-construction. However, like any language, OCaml has its limits. When writing low-latency code, you'd like the garbage collector not to kick in at inopportune moments. While this may seem like a niche use case, it is a niche that matters to Jane Street.

With OCaml 5, OCaml code can exploit shared-memory parallelism. Shared-memory parallel programming also leads to data races. Because noticing and debugging data races is difficult, code review isn’t sufficient to trust multicore code in mission-critical contexts. OxCaml extensions have three main goals:

1. Enable writing correct-by-construction multicore code;
2. Enable writing low-latency code in OCaml instead of a GC-free language;
3. Raise the overall level of performance.

Last but not least, all those goals should be achieved without moving OCaml away from its sweet spot.

Furthermore, according to its website, OxCaml’s primary design goals are to be “safe, convenient, [and] predictable”. Safety makes developers more productive and ensures they ship correct code. Convenience means using OCaml’s type system and type inference to provide adequate choice and control without adding complexity. Predictability involves retaining the aspects of OCaml that make it easy for developers to understand how their code will perform simply by looking at it, which requires keeping performance details explicit at the type level.

The OxCaml branch is open-source and welcomes new users. However, the extensions are experimental and not guaranteed to be stable or backwards compatible. Your feedback is needed to fine-tune the experience and improve the various new features. You can provide feedback by making issues on the OxCaml GitHub repo or discuss it in the #oxcaml channel in the OCaml community discord server.

Open-source tools and libraries provided by Jane Street now come in two flavours. The default targets classic OCaml. The with-extensions branch targets OxCaml. For instance, Jane Street's successful Base library has a with-extensions branch. Jane Street also provides an opam repository with OxCaml compatibility patched versions of the packages.

How Tarides is Helping

Tarides takes part in the processes around OxCaml in two ways: by providing platform support for working with and distributing OxCaml code, and by helping to upstream some of its features into mainline OCaml.

Tarides has adapted OxCaml features to the official compiler codebase and taken part in design discussions with the OCaml maintainers to ensure that the features integrate well into the existing compiler. Sometimes, the upstream compiler will have features that OxCaml doesn’t have yet, which overlap with OxCaml’s extensions, so care is needed to handle everything as seamlessly as possible and in a backwards-compatible way.

The OCaml 5.4 features labelled tuples and immutable arrays are two recent examples which have been upstreamed from OxCaml with Tarides’s assistance. For future releases, Tarides will be part of the design discussions for upstreaming new features, like "include functor", polymorphic parameters, module strengthening, and possibly more.

Experimental Extensions

Let’s take a look at the different extensions that are part of OxCaml:

• Modes: Modes are deep properties of values that are tracked by the OxCaml compiler. They are similar but distinct from types. While types describe what a value is, modes describe how they can be used. Each value in OxCaml has a mode (plus its type). OxCaml permits type signatures to be decorated with modes, restricting how that value may be used. As an example, the linear modality on a function says that the function may be invoked at-most-once. This property is distinct from the type of the function.
• Stack Allocations: With OxCaml, more values can be allocated on the stack instead of on the heap, which improves performance by reusing cache lines and reducing the cache footprint. The compiler uses a value’s locality to determine whether it is local and, therefore, can be allocated on the stack or global and needs to go on the heap.
• Unboxed Types: This extension gives users more options on how their data is represented in memory and registers. Unboxed types introduce the concept of layouts. Every type has a layout, with a number of base layouts available to the type system.
• Data-Race-Free Parallelism: There are a number of new features centred on OCaml’s support for multiple domains, including extending the mode system to track the concurrent use of values to improve safety during concurrency and introducing higher-level parallelism primitives.
• Kinds: This adds a new system that extends the type system by adding “types” to types, that's what a kind is, a type's “type”. Two words are used to avoid confusion, as kinds don't have allocated values at runtime; only types do. Kinds have several components, among them layout, which describes the shape of the data at runtime and modal bounds, which assign limits on the mode's value may be assigned to.
• Uniqueness: A mode that designates values that should only have a single reference pointing to them. By guaranteeing that only one reference will be consumed, the mode prevents bugs like the use-after-free segfault. There’s a Jane Street blog post, as well as a post by KC Sivaramakrishnan about using Uniqueness and its features.
• Comprehensions: This extension introduces ‘comprehensions’, which is a syntactic form that uses mathematical set-builder notation to build lists and arrays.
• Other Extensions: There are several more smaller extensions as part of OxCaml. These include immutable arrays, labelled tuples, polymorphic parameters, and more!

Looking to the Future

In light of the public release, we encourage OCaml users to try OxCaml and share their feedback. The best place to discuss OxCaml is the #oxcaml channel on the OCaml community discord server. Engineers from Jane Street and Tarides working on OxCaml regularly hang out in the channel and would be delighted to hear feedback.

The explicit goal of OxCaml is to iterate over these features and eventually upstream them to OCaml. Tarides has experience shepherding and successfully upstreaming Multicore OCaml, which was a multi-year effort to bring in native support for concurrency and parallelism to OCaml. Just as with Multicore OCaml, Tarides' goal is to make it easy for the community to work with OxCaml, gather feedback, iterate on the design with Jane Street engineers and help upstream the features to OCaml over the coming years. If we are successful with the upstreaming efforts, we believe that OCaml will fill an important gap in the design space for programming languages. We would like you to be part of this effort!

Stay in Touch

You can connect with us on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects. We look forward to hearing from you!

Background: Improving the OCaml 5 Memory Profiler

First, a little background on the area. In OCaml, we have support for statistical memory profiling built into the runtime, called statmemprof. The basic idea is the OCaml runtime provides an interface for registering callbacks to be called when interesting Garbage Collection events occur, such that we can track memory allocation activity for some statistical sample of allocations in an OCaml program. PR #12923 has more technical details of the implementation.

Built on top of this is memtrace, a library that uses the statmemprof interface to produce trace files formatted in the Common Trace Format (CTF). Memtrace has a detailed technical description of how it works. Finally, there is a web app, memtrace_viewer, that displays information about memory allocations using a 'FlameGraph' format to visualise the allocations. Below is a sample of what such a trace might look like.

For Kashish's internship we thought about how to support other kinds of visualisations for memory profiling data. For example, Go uses a directed graph visualisation in pprof that would be a good alternative to FlameGraphs. FlameGraphs are excellent for visualising data. However, they lack a useful property called join points, which are points where stack traces start differently and then reach the same important function. Using a graph representation highlights these points in a way that FlameGraphs cannot.

File Formats

A picture is worth a thousand words, and this is never more appropriate than when trying to understand a bunch of numbers collected from a complex system like a garbage collector (GC). We had two realisations in approaching this problem: one, that we could reuse the work done by others in visualising it, and second, that there are common tracing formats already used by other languages that we could reuse to unlock more visualisation options. For example, Memtrace used a CTF file format that looked like it could be converted to the protocol buffers (protobuf) based format used by pprof.

The first step towards converting formats was to understand the protobuf-based format. Looking at both the README.md and the profile.proto file gave an initial idea of the data types we needed. The pprof format is divided into three main parts: a Profile with general information, Samples recording values encountered in the execution of some program, and Locations identifying places within a program where samples are generated. Overall, the format is quite flexible and covers things like time profiling as well as memory profiling, which we're interested in.

Kashish looked at the pprof files generated from Go and Rust when doing memory profiling to see how they formatted their Samples. With this information, she started writing a tool to convert CTF traces into pprof traces. Starting with the profile.proto, she used pbrt and ocaml-protoc to generate the code for reading and writing the protobuf format; she then worked through the details of converting between the two formats. The end result is a cli tool for converting CTF files.

These files can be visualised using Go lang's pprof tool by running pprof -http localhost:8080 <proto_file>. For example:

The picture is zoomed in on the opamVersionCompare function, which represents 33% of the allocations in this solver-service program.

From the top left menu, you can choose Sample to view the graph by the ‘number of samples’, i.e., the number of times a particular stack trace occurs in our profile or ‘alloc size’, i.e., the amount of memory allocated by each stack frame.

You can also use Peek to see a breakdown of allocations sorted by space allocated or objects allocated. In GC parlance, an object is a dynamically allocated piece of memory that contains an OCaml value. This visualisation can highlight the top allocating locations in an OCaml program. This first image shows the top allocating functions based on the memory size of their allocations and highlights code in OpamFormat parsing that is worth investigating.

This second image shows the top object allocating functions unrelated to the amount of memory being allocated, which can highlight places that allocate many small pieces of memory and often go unnoticed. Note this again highlights the opamVersionCompare function.

A Library for Mappings: Blind Alleys

The pprof format contains a field called Mappings, which uses information about the executable binary and the virtual addresses of functions or Locations in our stack traces. We thought we needed to get the virtual memory address for symbols to include as part of the stack traces in the profile. This would allow users to map addresses back to locations in the source code via the addresses in the executable.

However, these virtual addresses are not known until runtime, so we needed to get the virtual memory maps for the running process. On Linux, this information was available in the proc filesystem as /proc/<pid>/maps and simply required parsing out the information into a usable type. Then, we mapped the virtual addresses to a segment and back to locations in the original binary.

On MacOS, things are much more exciting and tricky. On the surface, macOS is a Unix operating system based on a FreeBSD userspace; however, under this facade is another operating system called Mach, which is actually responsible for many aspects of the system, including how virtual memory maps for a process are represented. So, what we needed to do was write some low-level OCaml using c-types to call the right C functions to retrieve the information we needed. How that works deserves its own blog post, but you can read the code at tmcgilchrist/mach.

Later on, we realised that pprof traces could work in two ways; with memory addresses for compiled languages like C++ or Go or with symbolised locations for interpreted or JIT'd languages like Java or Python. The documentation calls these Unsymbolized profiles and Symbolized profiles respectively. OCaml will produce Symbolized profiles by default as Statmemprof supplies the location information for symbols. In future, Unsymbolized profiles could be supported, and even information like demangled names and source file locations could be included.

Writing pprof Directly

With the conversion code written and confident in our understanding of the file format, the next task was to write protobuf files directly from memtrace using the callback API provided by Gc.Memprof.start and creating a Gc.Memprof.tracker record. The resulting code was similar to the conversion code; however, there are some interesting points of difference for protobuf traces.

Protobuf traces tend to be larger than the equivalent CTF traces. This is because memtrace optimises the data written to a trace. The most important optimisation is the way it stores callstacks, as they are the single biggest piece of information stored. Consecutive backtraces usually only differ by the last few entries, so instead of storing the entire callstack each time, they store a “common prefix”, i.e., the number of entries that are the same as the callstack of the previous sample. Then, the reader can obtain the entire callstack by combining the “common prefix” entries with the new, unique entries.

With our restriction to use pprof for visualisations, we needed to use the pprof format as defined, and the pprof format does not support common prefixes. Meaning we could not implement common prefixes in our writer. In the future, it could be possible to write our own protobuf decoder that supports this feature.

To produce smaller trace files, pprof compresses its protobuf files using gzip, which significantly reduces their file size, making them much smaller than CTF files. To similarly reduce memory overhead in the protobuf writer, one option is to integrate an OCaml compression library such as camlzip to compress the output on the fly as data is written. Naturally, this introduces a trade-off: lower memory usage at the cost of increased CPU time. For example, when profiling a sample program, the CTF file is 25Mb versus 421Mb for protobuf, which reduces to 8.3Mb when gzipped.

The Go and OCaml Garbage Collectors differ in important ways that impact the information collected in trace files. Since the premise of this work is reusing the Go tooling and visualisations, it is useful to understand what kind of Garbage Collector Go uses.

Go uses a Tracing Garbage Collector with the following properties:

• Hybrid stop-the-world/concurrent collector
• Stop-the-world limited by a deadline (10 ms)
• Concurrent collector running in parallel on CPU cores
• Tri-colour mark-and-sweep algorithm
• Non-generational
• Non-compacting

Of these the most important property is Go's GC isn't generational, while OCaml's is a generational GC. Memtrace tracks deallocation and promotion events between the generations. Currently, we're not tracking these events as pprof was built for Go programs and doesn't handle this information. The next obvious step would be writing out these events and building the visualisations to handle them.

Collateral Fixes

In the process of writing tests for the conversion tool, Kashish also discovered some failing tests in the OCaml 5 version of Memtrace caused by the way the Gc.Memprof API interacts with threads and the new domains introduced in OCaml 5. They were fixed in this PR and will be included in the OCaml 5.3 support PR.

Until Next Time

The goal of the internship was to improve visualisation options for memory profiling in OCaml by investigating different profiling file formats and then build tooling to generate (or convert) to these formats. Pprof is a protocol buffers-based format used by Go, Rust, and Java to capture profiling information. Kashish built tooling for converting CTF trace files to protobuf format and writing protobuf format directly from memtrace, both of which allow users to visualise memory profiles using a directed graph format originally used by pprof.

In future, it would be interesting for the OCaml community to build on this work by:

• Extending protobuf format to record all OCaml GC events.
• Updating memtrace viewer to consume pprof format directly.
• Producing Unsymbolized profiles from stripped binaries (i.e. without symbol information and just addresses). Reconstituting symbolised information afterwards, similar to C++ or Go.
• Supporting encoding and decoding common prefix stack traces.
• Adding different kinds of visualisations like treemaps, force directed graphs, or circle pack layout

We would welcome contributions in these areas. Get in touch if anything there looks interesting or useful.

You can connect with us on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects. We look forward to hearing from you!

LLDB is the primary supported debugger for macOS and comes included with Xcode as part of Apple's developer tools. It supports both the ARM64 and AMD64 platforms, which OCaml also supports. Ensuring a smooth macOS experience is crucial, as it enables development on the Apple hardware used in the community and by Tarides engineers. This post provides an overview of the work done to enhance the macOS debugging experience for OCaml developers.

LLDB and Our Goals

Let's begin with some context about the technology we're discussing today. Debuggers are tools used to trace, manipulate, and visualise the state of a target program running on a target system. Developers use debuggers for tasks like tracing a program's control flow, inspecting the values of variables during execution, halting the program at predetermined locations, and executing functions within the running process.

Why focus on LLDB? It is a well-maintained project that supports a wide range of platforms we target, including macOS, iOS, FreeBSD, Windows, and Linux. Most significantly, for OCaml, LLDB is the only supported choice for ARM64 MacOS (an important and popular developer platform) and comes included with XCode. This means that providing a debugging experience on macOS requires LLDB. Unfortunately, GDB, another well-known open-source debugger, is unavailable for ARM64 macOS.

Based on our usage, we recognised that LLDB needed some attention and initially raised issue #12933, highlighting that setting breakpoints within LLDB was broken. Further investigation revealed other problems, such as printing backtraces producing incorrect results. Additionally, we saw the opportunity to integrate GDB features, such as printing OCaml values and running debugger tests within OCaml's test suite.

To make an impact with LLDB support, we focused on:

• Fixing how to create breakpoints in LLDB
• Porting GDB's Python-based value printers to LLDB
• Improving the debugging information emitted by the OCaml compiler

Breakpoints and Name Mangling

Breakpoints are a common feature in debuggers; they allow developers to halt program execution when a specific piece of code is executed. LLDB provides several methods for creating breakpoints. Firstly, you can use a memory address, or secondly, you can specify the name of a function, or finally, use a combination of a filename and a line number. Happily, using memory addresses to create breakpoints worked, but the other two ways were broken.

To understand how LLDB can set a breakpoint using a function name, we must explain how the compiler treats source-level names in OCaml and how that impacts LLDB. In the OCaml compiler, there is a process called name mangling, which involves turning the name of a program entity in OCaml into a form that is unique and can be linked against. Often, there are repeated names for particular functions or even modules in an OCaml codebase, and the compiler needs to generate unique names for them before sending them all to the linker, which is responsible for producing the final executable as a Mach-O binary.

Concretely, when setting a breakpoint based on a function name, it is necessary to use the mangled name produced by OCaml. During OCaml 5 development (while fixing a linking bug), the name mangler was changed to generate names like camlModule.function_name. Let's illustrate how this works with an example, consider this Fibonacci program:

(* fib.ml *)
let rec fib n =
 if n = 0 then 0
 else if n = 1 then 1
 else fib (n-1) + fib (n-2)

let main () =
 let r = fib 20 in
 Printf.printf "fib(20) = %d" r

let _ = main ()

The main function would get the name camlFib.main_123, and the fib function would be camlFib.fib_271; note that the trailing number can change between different runs of the compiler. These names are then used to create breakpoints within LLDB. Tab completion helps out, allowing you to specify part of the name and use tab to fill in the rest.

Unfortunately, using . in these names conflicted with LLDB, which used . for other purposes and wouldn't accept names with it present. Rather than modifying LLDB itself, we needed to alter the separator used in mangled names.

A one-character change seems simple, right? The other program that consumes these mangled names is called a linker, and they have restrictions on what characters can be used in a name, often restricted to a printable subset of ASCII characters. This problem already appeared in the MSVC porting work, where the linker on that platform wouldn't accept . either and a workaround was introduced to use $ instead. That was the approach we decided to take, and we modified name mangling for all platforms to use $. More details about how this change impacts other areas are detailed in PR #13050. This change will appear in OCaml 5.4, fixing the problem of setting breakpoints using mangled names and providing consistent names across platforms.

Setting breakpoints based on filename and line number was a more exciting experience, which we will cover in a later post.

Printing OCaml Values Using Python

OCaml uses a uniform memory representation in which all values can be kept in a single machine word, typically 64 bits on modern hardware. An OCaml value is either an immediate integer or a pointer to some other memory representing the value. This representation has implications for debuggers, as understanding how to print an OCaml value requires familiarity with this memory structure.

Both GDB and LLDB can be extended using Python as a scripting language. This capability enables developers to implement custom printing formats for values, add new commands, and perform various other useful functions. There were pre-existing macros for GDB for printing OCaml values, these could be rewritten into Python which would allow them to be used with both debuggers.

The resulting core ocaml.py library understands OCaml's uniform memory representation and how to print out values. The GDB-specific file gdb.py handles integrating with GDB's value printer, and a similar lldb.py exists for LLDB. The previous GDB macro file was retained for backward compatibility, but now it prints a deprecated warning when used. Beyond the core printing functionality, the new system also introduces improved commands ocaml and ocaml find, the former of which is introduced with the PR #13136 and allows for future sub-commands, and the latter being the heap search command, which was changed from gdb-macros.

Check out PR #13136 for more details, including several examples of what formatting you can expect when working with the debuggers. The end result is a Python-based solution shared between the two debuggers that can be easily extended in future.

Improving the Debugging Information

Debugging information is any data that is required by the debugger to perform its task. Often, this is extra information outside of the main executable. For example, a debugger needs to associate machine code in an executable with the source code used to produce it. DWARF is one such debug information format used on macOS and Linux systems.

We identified two issues with the debug information produced by OCaml:

1. Printing backtraces produced incorrect results
2. LLDB would not display the OCaml source for an executable

A backtrace is a visual representation of the current call stack for a program. There are two ways a debugger generates a backtrace (a process called unwinding): Call Frame Information (CFI) or Frame Pointers. CFI is part of the larger DWARF specification and is already used in the OCaml compiler. Clearly, the team needed to start by understanding the CFI information emitted by the compiler and validating the fixes. What followed was a series of PRs improving CFI.

The first PR, #13079, focused on fixing the backtraces for macOS on the ARM64 platform. Somewhere in the 5.1.1 update, a change happened to the CFI information OCaml produced that caused LLDB to lose part of the stack trace when moving between C and OCaml frames.

The bug was caused by the difference in handling frame pointers: C frames maintained them while OCaml frames did not. The OCaml code generator reused the x29 frame pointer register in the Iextcall fast path when calling from OCaml to C. Once identified, the fix involved correctly saving the x29 register, resulting in better backtraces for ARM64 on macOS, and Linux. #13595 is a follow-up bugfix for CFI-based backtraces, where the wrong CFA register was used. This part of the code was rewritten when adding frame pointer support for ARM64 on macOS and Linux #13500. Speaking of frame pointers, #13163 enabled frame pointers on the other macOS platform, AMD64. Now, printing backtraces can use either CFI or frame pointers to unwind the call stack.

The issue of LLDB not displaying OCaml source code for an executable was due to missing DWARF information, which is necessary on macOS but not on other platforms. We understand how to fix it and are working on a solution. Interestingly, the same lack of DWARF information is why setting breakpoints based on filenames and line numbers is broken.

Extras

While working on CFI for ARM64, we found and fixed CFI issues on other platforms. In particular, Linux for ARM64 and Risc-V platforms. The three PRs, #13241, #13261, and #13271, address incorrect Call Frame Information (CFI) for LLDB and GDB. These bugs highlighted the difficulty in ensuring the CFI information is correct and that debuggers work as expected. Although both GDB and LLDB use CFI, their interpretations of the specification sometimes differ, leading to subtle bugs.

To allow users to check their debugger's functionality and to help ensure that future changes don't break the debugging experience, the team enabled both the GDB and LLDB native debuggers to run as part of the OCaml test suite. Since both debuggers provide Python APIs to facilitate interaction between them and the test suite, programmers can write Python-scripted tests for their code. The PR #13199 has more details. Since merging this change, we have discovered a few bugs, such as #13509. The idea of writing a debugger test suite in Python is so good that it is what LLDB does, so we are in good company!

Finally, we gathered all the details we discovered about OCaml debugging and added a chapter to the OCaml manual documenting what we learnt. PR #13737 does just that, covering more technical details that might be interesting.

Until Next Time

Let us know your experience with debugging OCaml 5 programs! You can always share your feedback or questions on Discuss. For more information on how to debug OCaml with LLDB, check out Tim McGilchrist's blog post, which provides an excellent overview.

You can connect with us on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects. We look forward to hearing from you!

What Has Improved Since Last Time

If you check our tracking issue, you'll notice there are significantly more items there than there were before.

We've made enhancements in how dune pkg and the health check handle dependencies - including depexts - and aligned them better with how opam does it. There are, however, some intentional differences between how dune pkg and opam do things. Inspecting their repository health has led to fixes aligning dune pkg more with opam semantics and sometimes improving the correctness of the metadata on opam-repository. Below, we'll go through some of these improvements.

• A lot of packages in opam did not declare ocaml as a dependency. In theory, opam is OCaml-agnostic and can install packages written in any language (Topiary is written in Rust, for example). However, in practice, most packages on opam require OCaml, so when a package does not declare a dependency on OCaml, and none of its dependencies capture an OCaml compiler in their dependency cone, then Dune Package Management locks a solution without a compiler. In many cases, this will fail, so many packages have had their metadata updated to include ocaml as a dependency on opam-repository and, where possible, upstream.
• When opam encounters undefined variables, it evaluates them to 'false'. When locking a solution, we translate the build and install instructions into Dune's own variable format. However, in the Dune semantics, unknown variables are not evaluated as false by default. We changed the way we translate variables to wrap the variables with catch_undefined_var in #11512, thus matching the semantics of the original expressions.
• Some packages depend on ez-conf-lib, which is a package that records the place of its executable when it is built. Unfortunately, in the case of Dune package management, that would be a sandbox location, so when other packages attempted to access it, it would not exist at that location anymore. This made the package non-relocatable. In #11598, this was changed to use opam and Dune-provided variables, which are set to the appropriate location when building so that users can find them.
• When packages build, they often need additional dependencies from the operating system: these are called depexts. In Opam-Health-Check, we used opam to install these, but sometimes there was no valid solution, and opam would fail. Unfortunately, the failure displayed an error message, but the process still succeeded with exit code 0. We changed our code to detect the error message in #103 and ended up reporting the issue upstream to opam as #6488.
• When users locked a solution with dune pkg, it would also record the detected depexts. However, differences in how optional packages were handled between opam and dune could lead to not enough packages being installed if we used opam to install depexts. In #104, we changed the logic to use Dune to create the list of depexts and install these in a separate step. This way, there should be no confusion between what opam and Dune consider a dependency.
• While most source archives ship with .opam files, they are technically not required. Opam never reads them when installing (since it uses the information from opam-repository), and Dune does not need them as it can read all the required information from dune-project. However, Opam-Health-Check used them to determine which package names existed, so when it encountered packages without .opam files, it assumed there were no packages to build in the source archive. With #97, we read the package names from .opam files and from dune-project to ensure we capture all names.
• When opam builds packages with Dune, for the most part, it uses dune build -p <pkg-name>. The -p flag is a special flag which is mainly used for releasing and implies --release --only-packages <pkg-name>. We couldn't use the same -p flag, as --release itself expanded to a lot of other configuration options, among these --ignore-lock-dir. It meant that if dune pkg lock and then dune build were used, --release would ignore the lock directory. This was implemented so that introducing dune pkg would not break packages in opam-repository that used lock files. However, there aren't many packages in opam-repository that use --release and building packages with Dune package management in release mode is useful. Dune was patched in #11378 to move --ignore-lock-dir to -p. This allows you to use --release with package management, and #96 was merged to take advantage of it. The use of --release better represents an opam build and enables the building of several key packages, such as base and core, for which --release disables building Jane-Street-internal tests.
• When we looked for which packages to build, we accidentally used a subset search instead of an exact name match. Thus, we would sometimes accidentally pick packages to build that were not meant to be built. This was fixed in #99, ensuring that when determining whether lab should be built, accidentally matching on gitlab would not give us false positives.

Maybe Some Packages Just Don't Build

It turns out some packages that are on opam-repository just do not build. This can be due to a lot of reasons. Some packages don't support OCaml 5.3 (the most recent release at the time of writing and the one we run the checks on), and others don't support the platform we are running on. Some can't be downloaded because the server that hosted them disappeared. In such cases, there is nothing that Dune package management can do besides fail.

Thus, to make a fair comparison, we patched Opam-Health-Check and extended it so it can build the same package with Dune and opam in the same run. That way, we see that if a package doesn't build on opam, it is unlikely to magically work when using Dune package management (although that can happen, e.g. on transient network failures, which would prevent opam from downloading the source tarball).

Some Things We Don't Support

There are some packages that will not work. Often, this is because the packages fail due to how Opam-Health-Check works, which is not something we expect a user of Dune package management to encounter.

Complex Build Commands

When selecting the packages that we plan to build, we make sure to only pick Dune packages. However, the definition of a package 'using Dune' is not clear-cut.

A source might have a dune-project file but never call Dune. A build might call Dune but also do an arbitrary number of other steps. In opam, this process is simple because opam will just execute all steps in the build and install entries, be it launching Dune, calling make, or any other command.

For the health check, we decided to set the limit at dune build. This means that packages that require extra instructions will most likely fail to build in the health check.

The reason why we are setting the limit here is twofold:

1. Interpreting which commands to run in the health-check would require us to implement and evaluate the filters that opam supports for running the commands. Making sure we evaluate things exactly like opam does would be a non-trivial undertaking.
2. Packages that need extra commands to run usually run just fine when these commands are run manually; thus users of dune pkg can most likely use Dune package management when using it on their machines.

Another bonus reason is that not that many packages are affected by this, so it didn't seem worth the time investment.

What Work is There Still Left to Do?

There are still categories of errors that make it difficult to adopt package management. The most notorious issue is #10855, colloquially called the "in and out of workspace" bug.

It occurs when a project has dependencies, and these dependencies, in turn, depend on a package that is in the project's workspace. Usually, this is a circular dependency, but such a configuration can reasonably happen in some cases, such as when a test dependency uses something from your project. For example, if Lwt uses a test tool that, in turn, depends on Lwt, it is currently impossible to build it with Dune package management, as Lwt would be part of both the build and its own dependencies.

There are not many packages affected, but the ones that are are some of the most used packages in OCaml. Among these are Lwt, Odoc, and, unfortunately, Dune (due to lots of projects depending on dune-configurator). Thus, at the moment, Dune package management cannot be used to develop Dune itself.

While addressing these issues was outside the scope of this particular project, we plan to tackle them through future initiatives. Ultimately, our goal is to provide a seamless user experience with Dune package management.

Until Next Time

If you're using Dune Package Management and have feedback or questions, please share your thoughts on Discuss. Our teams are always looking for input in order to improve tools and features, and your feedback can help us make everyone's experience better.

Stay in touch with Tarides on Bluesky, Mastodon, Threads, and LinkedIn. We look forward to hearing from you!

The kick-off of the CEOS (data Center and Edge cOmputing in Space) project was March 24, we're thrilled to be part of it and to be in such good company. CEOS is a collaboration between STEEL Electronique, ELSYS Design, IP-Maker, Thales Alenia Space and Tarides to create a new generation of efficient, multi-purpose satellites. The project's overarching aim is to significantly diversify and expand the onboard capabilities of satellites to survey the environment. Tarides’ role is helping implement an efficient onboard OS and orchestrator that can dynamically deploy multiple earth observation applications on one satellite.

Let's take a closer look at the goals of CEOS and the technology making it all possible!

How is CEOS Addressing Current Limitations?

One of the best ways to support scientists, organisations, and governments in protecting the environment is to provide them with reliable, up-to-date data. A great way to source this data is to use Earth observation satellites, which can take images and make a multitude of measurements from orbit. Using two or more images of the same geographical area makes it possible to compare them and highlight the differences. The potential use cases are vast, including surveying for changes to biodiversity or deforestation and alerting users to anything from wildfires to natural disasters.

Usually, these images would need to be transferred to the ground before they’re processed, which can limit response times and saturate the communication link with mostly useless data (we are only interested in the changes and their nature). Alternatively, trying to process them on board would be a slow process on a normal satellite CPU, but the AI hardware and software upgrades the CEOS project brings to satellites will allow them to detect and analyse changes quickly and alert the relevant authorities or stakeholders. Using on-board machine learning to detect changes has big implications for efficiency and performance. If AI programs run in space, they can process and detect changes quickly enough to send near-instantaneous alerts to recipients on Earth. By enabling fast response times, this approach can save precious time in an emergency (for example, in the case of chemical spills or forest fires) and mitigate the damage. Furthermore, having multiple different applications available on-board (potentially running in parallel) allows satellites to have more diverse missions (for example switching from wildfire detection to detecting fishing abuses when the satellite moves from land to ocean) and avoid idle portions of the orbit.

Another major aspect of the CEOS project is the actual fabrication of the demonstrator board of the satellites, with STEEL responsible for its design and manufacturing. The board will be based on a Xilinx Versal SoC, and feature high-performance NVME storage (with IPs from IP-Maker) to store vast amounts of sensor data quickly and later process it. The satellites will be designed to be usable for multiple missions and have a longer service life, reducing the demand to produce new satellites and put them in orbit.

What Technologies are Being Developed?

Let's take a closer look at the technology we're developing alongside our partners, with the end goal of using AI to detect relevant changes earthside, deploying multiple AIs on one satellite with an FPGA architecture.

Detecting Changes

Using a computer to detect changes is not as straightforward as it may seem to us humans, who, after all, perform this type of calculation on a daily basis without thinking much of it. There are three ways to detect changes in computer programming:

1. Binary detection of changes: can only detect whether a change has occurred but not where
2. 'From-to' detection: monitors for a single, specified change between a base level and a later state.
3. Multi-class changes: several changes can be detected and classified according to their type

These algorithms can process several different types of images, including radar images and optic ones. Furthermore, we can use deep learning to validate the changes detected by the algorithms. For example, they should ensure that they haven't flagged a change in tree cover due to something as simple as a change of angle or a picture taken during a different time of day. By using deep learning, we can make the validation process much more efficient than if we rely on human intervention.

The CEOS demonstrator will embed several change-detection applications designed by ELSYS, the experts on the matter.

The Operating System

The operating system's and orchestrator's designs are also cutting-edge. In collaboration with the partners behind the European project ORCHIDE (in particular Thales Alenia Space), of which Tarides is one, a minimalist OS based on Linux and several lightweight orchestration components will be deployed on all boards. These components will allow the orchestration and execution of applications as isolated and light unikernels, mirroring SpaceOS.

SpaceOS, and in general the ORCHIDE project, takes a modern approach to operating systems by deploying an efficient, specialised, light-weight OS unique to each application. This reduces its dependency on a host architecture, and a simpler, less complex host structure like a hypervisor is enough. It also reduces bloat, with the system only strictly using the necessary resources, improving energy efficiency. Combined with state-of-the-art technologies for storage, orchestration, metrics collection and workflow design, ORCHIDE aims to provide a complete solution to design and embed complex dynamic payloads, such as the ones CEOS will handle, without compromising on efficiency and cybersecurity (thanks to the strong security guarantees of unikernels).

The challenge (and opportunity!) of this project will be to support a range of hardware acceleration devices like FPGAs and GPUs, in an agnostic way (as much as possible), and through the virtualisation barrier that is inherent to unikernels.

What's Next?

This exciting project has only just begun, and a lot of work has yet to be done over the next three years. To stay up-to-date on our progress, keep an eye on our blog and Parsimoni's website, where you can also learn more about SpaceOS.

The end goal is to take a multifaceted approach to helping the climate by giving scientists, organisations, and governments access to powerful tools to help them monitor and safeguard the environment. The implementation also promotes sustainable practices by encouraging the use of a single satellite for longer and for more than one purpose.

Stay in touch with Tarides on Bluesky, Mastodon, Threads, and LinkedIn. We look forward to hearing from you!

Functional programming fits in well with the web world. The transactional nature of HTTP and the convergence towards immutable state management solutions (such as Redux and consort) make OCaml a very good candidate for web application development. In this article, we give you an overview of some of the web solutions supported by the OCaml ecosystem. It's important to note that we can't include every single one of the web development libraries available for OCaml, as there are simply too many for one post!

Functional Programming, OCaml, and the Web

Why do some developers use functional programming for web development? Functional programs leverage concepts like immutability, higher-order functions, and formal verification to achieve, among other beneficial outcomes, better code reusability, parallelism, and fewer bugs. These developer-friendly qualities that simplify programming and increase productivity are behind the rising interest in FP for front- and backend web development.

As a functional programming language, OCaml offers great scalability for user projects by making it easy to build large systems without sacrificing maintainability. Thanks to the expressive type system, it is easy to refactor and adjust projects during development and as requirements change and evolve. The type-checker is reliable and covers 100% of the code, making it easier for developers to verify that data is used correctly in operations. With these features and the general benefits of FP, we're noticing a growing interest among software developers using OCaml to create web applications.

There are several tools and libraries designed for web development in OCaml and a lot is possible with the language. Some workflows are still being developed, and many developers are building new projects and sharing them with the open-source community for feedback.

Before we begin, there are three notable industry success stories, BeSport, Ahrefs, and Routine. BeSport is a popular French sports social app built using Ocsigen, a full-stack modular web framework for OCaml that empowers developers to build websites, web applications, and even mobile applications. Ahrefs, the world-leading marketing intelligence platform, uses full-stack OCaml for their web stack and has been very successful migrating to Melange. Routine, an integrated work platform that organises all your data in one place, uses Dream and wasm_of_ocaml to run their tools. With these companies showing how to leverage OCaml's potential, let's move on to the tools and libraries that make it all possible!

Ocsigen: A Complete Framework

Ocsigen is a collection of projects that provide a complete framework for developing web and mobile apps in OCaml. It is suitable for various uses, from simple server-side websites to client-side programs and complex client-server applications. The projects included under its umbrella are: Lwt, a general purpose concurrency library for OCaml; TyXML, for generating typed XML; Js_of_ocaml, which compiles OCamly bytecode from JavasScript and WASM; Eliom, which is a multi-tier framework for client-server web and mobile apps; Ocsigen server, a web server; Ocsigen Toolkit, a client-server widget library for Eliom and Js_of_ocaml; and Ocsigen Start, a higher level library providing user management and an application template that can be used as a basis for apps or to learn. The projects are mostly independent of each other, so users can pick and mix what they are interested in.

Ocsigen's design focuses on a couple of main strengths. Firstly, it takes full advantage of OCaml's expressive type system to check multiple programs' properties at compile time. This approach drastically reduces the development time of complex apps and makes it easier to refactor an application's code to facilitate new features. Secondly, using the Eliom framework enables multi-tier (also known as universal) programming where an application's client and server side are written using the same language and as a single code, with annotations in the code to indicate which 'side' the code should be run. As a result, both the server and client parts of a web application can be implemented as one program.

This method affords the developer a lot of flexibility to, for example, generate parts of pages either in the client or server code, depending on their needs. It also streamlines communication between server and client since programmers can use server-side variables in client code or call server-side OCaml functions from client code. It also makes it easy to deploy web and mobile multi-platform applications. Android and iOS apps are generated from the same exact code of the web app and run in a web view.

There are loads of resources available online to learn more about Ocsigen. Check out this presentation on Watch OCaml for an overview of the framework. For an example of what a mobile app developed using Ocsigen looks like, you can download the BeSport app on Google Play or the Apple app store.

Backends

Backend web development creates the foundations of the web, being the server-side portion of the website you don't see that implements the functionality of what you do see in your web browser. Common languages include Python, Ruby, and Java, just to name a few. OCaml has several web frameworks that enable you to do backend development, including options for beginners (e.g. Dream) as well as more experienced software engineers (e.g. Ocsigen).

Dream

Dream is a backend for OCaml that is well-liked for its simplicity and minimalist approach. Developer experience is a central tenet of its design, achieved by having a simple API and relying on fundamental OCaml types like string and list, only introducing a few of its own types. For a newcomer, the web framework offers extensive documentation and plenty of examples to get them started. Further examples of its quality-of-life features include unified error handling, a simple logger, a minimalist programming model that lets the developer create web servers just using functions, and cryptography helpers and key rotations to set up security options.

The Dream web framework is composed of several sub-libraries with different dependencies, which allow the user to port their projects to a variety of environments according to their needs. Furthermore, Dream is unopinionated and low-level, letting users pick and choose how they want to use it. They can swap out libraries to use other tools instead of its built-in templates (such as mlx). Essentially, the framework aims to be easy to use but highly configurable, giving the user the choice between customisation and simplicity. More concretely, Dream provides HTML templates (for OCaml or Reason); helpers for secure cookies and CSRF-safe forms; easy HTTPS, HTTP/2 and WebSockets support (meaning it supports most modern Web transport protocols); full-stack ML with clients compiled to Melange, ReScript, or Js_of_ocaml – and more!

If you want to try Dream, check out the tutorials. We also recommend this blog post on the Ceramic Hacker website, part two in an OCaml web development series that covers using Dream as a backend. It has some concrete code examples and gives a nice overview of a project. Of course Dream's homepage is also an incredibly useful resource.

Interop With JS, TS, and Wasm

JavaScript is the reigning monarch of programming languages for the web, powering web servers and adding interactivity and dynamic elements to web pages. With a vast array of tools and features in a mature ecosystem, JavaScript cross-compatibility is a must for languages aiming to expand onto the web. Wasm, or WebAssembly, on the other hand, is a portable compilation target that enables deployment on a variety of platforms. It is popular for its security guarantees, speed, and language- and platform- neutrality.

Js_of_ocaml

Js_of_ocaml compiles OCaml bytecode into JavaScript. This allows you to create dynamic and interactive elements on web pages and use tools like Node.js. You can install js_of_ocaml using opam or Dune, with the latter providing native support.

Some of the benefits of js_of_ocaml include its ease of use; it is simple to install and works with an existing installation of OCaml without requiring you to recompile your libraries. It also comes with existing bindings for many browser APIs and is stable and easy to maintain. Plus, performance comparisons have indicated that js_of_ocaml typically outperforms the OCaml type code interpreter. Furthermore, by generating JavaScript from OCaml bytecode, js_of_ocaml relies on a very stable interface that allows it to easily remain compatible with new compiler releases and most of the OCaml ecosystem.

To try it out yourself, there's a great js_of_ocaml tutorial by Jack Strand showing you how to create an interactive animation for a website. There are also some examples of js_of_ocaml in action, like this animated 3D view of the earth and a Boulder Dash style game.

Wasm_of_ocaml

Wasm_of_ocaml takes OCaml bytecode and transforms it into Wasm code. Originally forked from the js_of_ocaml compiler (and now merged back) it is of a similar, lightweight design. WebAssembly provides a sandboxed environment and enforces memory safety, making it popular among users who develop mission- and security-critical applications. Wasm_of_ocaml uses the WebAssembly garbage collection extension, removing the need to implement a garbage collector through other means and enabling good interoperability with JavaScript.

One of the compiler's biggest benefits is its speed, with some very impressive results from recent benchmarks driving renewed excitement. Compared to js_of_ocaml, programs compiled with wasm_of_ocaml are consistently faster. For example, Jane Street reported that they observed 2x-8x performance improvements using wasm_of_ocaml compared to js_of_ocaml. You can learn more about wasm_of_ocaml from our blog and in the repo's readme.

Melange

Melange is another backend for OCaml, consisting of a set of tools that makes it capable of generating and interacting with JavaScript. The tools include a compiler and compiler libraries, which can generate JavaScript code, and the runtime, which consists of a series of supporting libraries written in JS which output 100% JS. This makes interoperability and incremental adoption much easier for new users, albeit at the small cost of needing to write OCaml or Reason in a slightly different way.

One of Melange's biggest strengths is that it provides support for many different tools such as the editors VSCode, Vim, and Emacs, full integration with the very popular build system Dune, and interoperability with the documentation tool odoc which can generate a diverse range of documentation from the user's files. Overall, Melange's integration with the OCaml Platform gives the user access to the great resources of the OCaml Ecosystem.

Melange integrates with JavaScript through an expressive bindings language. This opens up the JavaScript ecosystem to the OCaml programmer, allowing them to use existing JavaScript packages and their own JavaScript libraries in OCaml projects and build applications that rely on features from the JavaScript ecosystem by working well with the syntax extension format ppx, Melange benefits from its performance, functionalities, and compatibilities.

You can try Melange in the Melange Playground and learn more about its finer details on Melange's documentation website.

For those familiar with ReactJS, there is an excellent online resource specifically aimed at React developers who want to learn Melange. It provides a hands-on introduction to Melange with several projects and examples. Melange offers excellent support for React codebases, with clear JS outputs and first-class support for many React patterns. David Sancho, one of Melange's maintainers, regularly produces very good content on his blog about the use of Melange in real-life applications, for example: Server-side rendering React in OCaml. This elegant interoperability with ReactJS demonstrates Melange's desire to interface easily with the JavaScript ecosystem, making OCaml a serious replacement for TypeScript, while continuing to benefit from the vast ecosystem of the JavaScript world!

Frontend

Frontend web development refers to the creation and editing of the graphical user interface of a website, including everything pertaining from HTML editing and rendering to web design, up to client-side web applications that run entirely in the browser. The underlying languages for frontend web development include HTML, cascading style sheets or CSS, JavaScript and WebAssembly. There are also popular platforms for frontend web development, such as Wordpress.

OCaml-VDom

The ocaml-vdom library, developed by Lexifi, implements an Elm-like architecture and VDOM for OCaml. An Elm architecture refers to a development pattern that is used to create interactive programs. The Elm architecture is a reformulation of a Moore machine, adapted to the construction of user interfaces. It enables the state of an application to be controlled using 3 ingredients:

• A Model: the state of the application,
• A View: A function which takes a model and returns the representation of the UI (in the case of Elm and OCaml-vdom, an HTML document); this document allows messages to be propagated, triggering the final ingredient,
• An Update: A function that takes a Message (propagated by the View) and the current model as arguments and computes a new model, which will be passed to the view to rebuild the UI.

In practice, the Elm architecture also introduces the notion of Command and Subscription to interact with discrete effects and communicate with the outside world. This approach is very expressive for describing reactive interfaces and is the result of several iterations described by the creator of Elm in the following essay: Elm: Concurrent FRP for Functional GUIs. After much experimentation, he has drastically simplified the user experience of Functional Reactive Programming with the Elm architecture!

OCaml-vdom is a very faithful implementation of the Elm architecture, giving OCaml the ease and expressiveness to describe rich web interfaces. To explore some examples and test OCaml-VDom yourself, visit the ocaml-vdom library and get hacking!

Bonsai

Bonsai is a client-side web framework created by Jane Street that lets users build web pages from the client (the browser) by creating components. For those familiar with other web development frameworks, Bonsai fills the same role as, for example, Angular and Vue, but it is written entirely in OCaml.

The structure of Bonsai is as a group of components, each representing one part of the final page, available in the bonsai_web_components repo. One of the benefits of the framework's design is that, instead of structuring its components like a tree, it does so in a "structured acyclic graph". This lets programmers create components that can 'communicate' with each other much more easily than with frameworks that use the tree structure.

To learn more about the framework, we definitely recommend the same article series mentioned above from the 'Ceramic Hacker' blog, especially starting from the Setting up Bonsai post. In it, Alexander Skvortsov illustrates why he found Bonsai excellent for building "safe, massively scalable, performant UIs".

… And Many More

There are plenty more libraries available for web development in OCaml, and we have not been able to include them all in this post. For example, Brr provides a toolkit for programming browsers in OCaml using Js_of_ocaml; the Lwd library enables reactive programming in the browser through a simple form of incremental computation; Vif is an experimental program that runs an OCaml script and launches a web server from it; React is a functional reactive programming library for OCaml; ReScript is a typed language that grew out of OCaml, now used to write fast and human-readable JavaScript for the web; and Wasocaml is another WebAssembly compiler developed by OCamlPro which works differently than Wasm_of_ocaml.

Furthermore, several big OCaml projects were developed in connection with extensive research done at universities, including projects like Ocsigen and MirageOS (check out Unipi for a great example of a static web server built with MirageOS). For example, 'Rethinking Traditional Web Interaction' is an interesting paper exploring the future of web development far ahead of its time, research which directly impacted Ocsigen's design!

While this post has focussed on application development frameworks, MirageOS offers a robust, mature, and efficient deployment solution similarly grown out of a rich academic history. When deploying a traditional application, it is common to separate it into multiple execution units (i.e., microservices). However, each of these services requires an OS (and ideally containerisation). MirageOS is a framework that allows developers to define an operating system (executed by the CPU's hypervisor) that only runs a limited set of tasks: a unikernel. This enables the creation of very lightweight operating systems designed to perform only a restricted set of tasks, significantly reducing the attack surface and boot time. As a result, MirageOS provides an advanced set of tools for development using microservices.

Since OCaml is an excellent choice for building compilers and build systems, there are many advanced static web site generators to choose from! Without going into too much detail, we'd like to mention Soupault, a highly flexible HTML processor that offers a great deal of freedom when it comes to building static pages; Stog, a mature tool with a huge number of plugins; and YOCaml, a very generic framework for composing your own static site generator. There are many others, and since a static site generator is a specialised version of a build system, Dune can even be used as one! In any case, OCaml is a useful (and fun) choice for creating a static site, which also works well alongside MirageOS for deploying very small static servers or using Git as a static file system (directly conceivable with Yocaml_git).

These are just a few examples of the things we didn't cover here, and there are many more great projects out there to discover. If you think we've missed one, please let us know!

Until Next Time

Thank you for checking out this article on web development in OCaml. We have tried to capture as much information as possible in one place (as you can probably guess from the length of this post!) and we look forward to hearing your feedback. Have you developed any web applications using OCaml? What was your experience? Connect with us on Bluesky, Mastodon, Threads, and LinkedIn. We look forward to hearing from you!

The convergence has been so strong that, over the years, some big names from the OCaml community have shown up — like Anil Madhavapeddy, Hannes Mehnert, Gabriel Scherer, and even Xavier Leroy! This is one of the many reasons why Tarides decided to sponsor the 2024 edition and send Sabine Schmaltz and Leandro Ostera.

For the 2025 edition, March 14, 2025, in Berlin, Xavier Van de Woestyne had the privilege of presenting Tarides’ work on editor support for OCaml during his talk "Beyond the Basics of LSP: Advanced IDE Services for OCaml." Accompanied by Antonin Decimo, who attended the conference, Xavier travelled to BOBKonf 2025 to share their insights and experience.

A Wide Range of Interesting Talks

The BOB program is wonderfully eclectic, and every talk is an opportunity to discover something new! For example, after a keynote on Local-First software — which included many fascinating use cases with potential applications for Irmin. We had the chance to attend talks on abstraction, speculative reasoning about functions based on their types (for instance, a function of type a -> a having only one possible inhabitant), the application of separation logic for concurrency in Idris, and even collaborations between engineers and mathematicians on the specification of formal methods.

We explored the functional programming counterpart to design patterns — with a strong emphasis on the power of robust module systems, something that deeply resonates with us as OCaml developers. That was followed by a deep reflection on object-oriented programming from a functional programmer’s perspective, a clear explanation of how recursive definitions work in Lean, and, to wrap it all up, a guide to common pitfalls to avoid when building distributed systems with microservices.

All in all, it was an intense and inspiring day — packed with ideas that strongly resonated with us. From our perspective, the themes explored throughout the conference aligned closely with the ideological and technical choices we’ve made at Tarides, particularly our commitment to OCaml. But beyond that, many of the talks echoed the challenges and directions of the projects we actively maintain!

About our Presentation

Although the goal of our presentation (you can watch the recording on BOBKonf's website) was to discuss OCaml editor support (through Merlin, Ocaml-lsp-server, and its clients, Visual Studio Code and Emacs), we aimed to present an approach and a set of features that wouldn’t limit our audience to just OCaml users. Instead, we wanted to spark a conversation with other IDE users/maintainers to share ideas and implementation perspectives!

We believe the presentation was well-received, generating some very interesting questions along with positive conversations about how some of the ideas we presented could be applied to proof assistants like Isabelle, Idris2, and Agda.

There was a proposal to combine our efforts to improve the Language Server Protocol, making it even more welcoming for certain functional languages that leverage interactive features (where the acceptance model is primarily based on voting). From our perspective, these were excellent and motivating responses!

Meet and Greet

Beyond the technical side, one of the great things about conferences is the chance to meet people—catch up with familiar faces, make new connections, and have meaningful conversations around topics we’re all passionate about! From our perspective, even though the schedule is quite packed, the talk slots are spaced out just enough to let us catch our breath — but more importantly, to connect and chat with members of the community. It really helps to foster a friendly, sociable atmosphere!

To Conclude

Attending conferences is an integral part of our work as engineers—for several important reasons:

• Keeping up with the latest in technology and research
• Sharing our progress and presenting the work we’ve been doing
• Initiating potential collaborations with people driven by similar goals and motivations.

So yes, it's important — but at conferences like BOB, it’s also a real pleasure! The talks are truly fascinating (we’re already looking forward to the video recordings so we can catch up on what we missed), and the interactions are incredibly motivating for our work. If, like us, you’re interested in functional programming, fancy types, formal methods, and many other exciting topics, don’t hesitate to check out BOB’s YouTube channel – and maybe even consider attending next time!

Connect with Tarides online on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects.

In this post, we look at another returning property, MSVC support, and the steps along the path to implementation. If you want to skip straight to the code, check out the #12954 pull request (and the dozen more linked from it!) in the OCaml repo. Let's dive in!

MSVC Support for OCaml on Windows

First, let's explain what 'MSVC support' means. In general, OCaml supports compilation to Windows through three separate toolchains: Cygwin, mingw-w64, and MSVC. The mingw-w64 toolchain was available for OCaml 5 from the moment the update launched. Cygwin was restored in OCaml 5.1, but MSVC support has lagged behind until now!

The delay stemmed from MSVC's initial incompatibility with C11 atomics, which the OCaml 5 runtime requires. David Allsopp had been exploring possible ways to overcome this incompatibility, testing how C++ atomics worked with older compilers. Eventually, however, Microsoft introduced support, albeit experimentally.

To restore the port, the team needed to ensure that the C11 atomics support was reliable, port winpthreads onto MSVC, and create a continuous integration (CI) workflow.

Since it is Microsoft's own C/C++ compiler, MSVC is popular and well-known to many developers and Windows users. Bringing compatibility with the compiler to OCaml 5 is an important step towards enabling more users to adopt the latest version of OCaml and explore its new features!

C11 Atomics and Bug Reports

C11 is a version of the C standard. Since the OCaml runtime is written almost entirely in C, the general standard lets us specify which properties of the C compiler can be used to build OCaml. If we didn't rely on a standard, we would have to list supported C compiler versions individually (GCC from one such version, Clang from another such version, etc). Instead, developers know that the OCaml runtime environment supports any C compiler that is C11-compliant.

For OCaml 5 and onwards, the C compiler must be C11 compliant and support C11 atomics. All we need to know about atomics for this post is that the C11 atomic spec enables the compiler to help the programmer access data that can be shared between multiple cores. Without it, the developer would need to use other synchronisation mechanisms such as mutexes, that require more code, both to write and to run. So C11 atomics go beyond what most of us associate with atomicity and are key to writing sound and efficient code in a multicore setting.

Once the Visual Studio 2022 release introduced experimental support for C11 atomics, it provided a much clearer path for the team to work on restoring MSVC support. This team included David Allsopp, Antonin Décimo, and Samuel Hym from Tarides, but of course, the success of the project relied on the collaboration, input, and reviews of many open-source OCaml community members. With C11 atomics support in Visual Studio 2022 being experimental, the team needed to ensure that all the sequential tests passed and identify places where parallel tests failed due to bugs in MSVC. The team created several bug reports against the MSVC compiler as a result of this project.

The bug reports include:

• Missing Atomic Stores When Dereferencing Pointer to Atomics: MSVC version 19.38.33128 lacked support for pointers to atomic values and was not emitting atomic stores when writing to a dereferenced pointer for an atomic variable.
• Compound Assignment Operators are not Atomics on Pointers to Atomic Values: Again, MSVC 19.38.33128 lacked support for pointers to atomic values, and this bug report highlighted that MSVC did not generate atomic code for compound assignment operators with a pointer to an atomic value.
• Pointers to Atomic Values Should be Reloaded: In version 19.38.33128 MSVC failed to generate atomic code, meaning that pointers were not reloaded when they needed to be, causing threads to spin indefinitely.

Thanks to great support from Microsoft, these bugs were resolved, and C11 atomics support was satisfactory to enable MSVC support for multicore OCaml. This was the biggest roadblock to the project's success, and with it cleared, the team turned their attention to new challenges.

Winpthreads and MSVC

The next hurdle on the road to success was another Windows-specific compatibility issue. OCaml 4.* had limited support for threading in the form of the optional systhread library, but the runtime itself made no use of it. That completely changed with OCaml 5! The abstraction used to enable threading support was Unix's posix threads, known as 'pthreads'. At the time, the runtime was prepared in the hope that a Windows version could be implemented in the future.

However, the original multicore PR could use the winpthreads part of the mingw-w64 library to provide a pthread implementation for the Windows MinGW port. The intention then was that it would be a temporary workaround allowing all the existing pthreads code to be reused, partly due to the belief that it would only work for mingw-w64 and not for MSVC.

Upon further investigation, David discovered a library demonstrating that winpthreads could be compiled with MSVC without introducing too many dependencies. Samuel and Antonin worked on formalising the process of extracting the winpthreads sources from the mingw-w64 project to use them for the MSVC port. Antonin also contributed directly to the mingw-w64 project to patch its winpthreads component.

Thanks to this work, the initially temporary winpthreads workaround has been implemented as a submodule for MSVC. This lets the new MSVC port use pthread.h via the winpthreads submodule (instead of using winpthreads implicitly as provided by the mingw-w64 GCC compiler).

The future of pthreads in OCaml is still up for discussion, with one school of thought being that reimplementing OCaml's use of pthreads in a more abstract way would allow its primitives to function without the full weight of the POSIX spec, resulting in better performance. Work has started to remove winpthreads and use modern Windows APIs for the MSVC and MinGW-w64 ports.

CI

Finally, the team added a continuous integration workflow enabling GitHub Actions for MSVC testing.

As most OCaml compiler developers use a different port than MSVC, and since there are many differences between MSVC and the other ports, being able to test MSVC in CI helps the developer be confident that their modifications do not break the code. In particular, part of the CI workflow includes a check to make sure the assembly used in the MSVC port (meaning that it's written in MASM syntax) is kept consistent with the assembly used in the MinGW port (written in GNU Assembler syntax). This check has already allowed OCaml core developers to catch a few PRs that only updated the assembler in GNU syntax, catching a problem early and preventing it from affecting the program.

A Note on the Unloadable Runtime

Before we leave you, let's briefly review another feature parity project, the return of the unloadable runtime. This project was paired with the return of the MSVC port as two features that were important to the community. The 'unloadable runtime' is a feature that cleans up OCaml resources, including the stack, heap sections, code fragments, buffers, tables, and more, when OCaml is used as a shared library. For example, if a host program uses OCaml as a library, when control returns to the host program, the unloadable runtime ensures proper resource cleanup.

The return of this feature was requested by the community, and our team worked hard to make the restoration a reality. The PR associated with this effort is #12964, which you can check out to learn more about the process behind the changes. The PR has been merged and is expected to be released with the 5.4 update.

Stay in Touch!

Keep an eye out for future updates on restored features on our blog. For a broader overview of the 5.3 update, you can check out our release blog post covering the changes.

You can connect with us on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects. We look forward to hearing from you!

What do we mean by maturation? The goal is fuzzy (as with every software, it is never 'done'), but we want to get Dune package management into a shape where we can consistently recommend that people use it for their projects.  They should be confident that their workflows will continue to work while unlocking the new features that Dune package management brings.

The core points of this are:

• The OCaml Platform Tools should work at least as well with Dune package management as they work with opam. With the new features in Dune, this interoperability should work even better as users do not have to share dependencies with the project in the local switch since tools can be installed automatically, possibly even from precompiled binaries. Do you want MDX? Declare a dependency, and voila, you have MDX.
• Most projects can start using Dune with little to no adjustments. The majority will work out of the box, and the most frequent fix required is to correct the list of project dependencies. No substantial code changes are necessary, and all projects should continue to be compatible with both opam and dune; there is no lock-in to one tool or the other.

Our goal is to successfully build as many projects as possible using Dune's package management feature. But to evaluate what we have left to do, we need to know where we stand now. This blog post will give you an overview of the project's scope and biggest challenges. 

Building "All" Packages

What if we want to try to build all the existing OCaml packages? opam-repository to the rescue! While it might not include proprietary code bases, there are still a significant number of projects we can try to build with it. Fortunately, there has already been prior work done on this subject. Opam-health-check is an existing tool mostly written by Kate that can determine whether packages can be installed on different historical, current, and future OCaml versions. It continuously monitors the state of the opam ecosystem, which inspired its name.

Tarides is running and maintaining multiple opam-health-check instances for the community. The most well-known is check.ci.ocaml.org which regularly builds thousands of opam packages on Linux, freebsd.check.ci.dev which does the same thing but on FreeBSD, and windows.check.ci.dev which as the name implies builds packages on Windows to help us with the effort to deliver a better OCaml experience on Windows.

We were wondering whether we could use the tool when building with Dune instead of opam. Fortunately, the software is free, so we could extend the functionality to build Dune projects instead of installing opam packages. This gave rise to the next instance of opam-health-check, dune.check.ci.dev which, instead of using opam, builds them using Dune package management.

Which Packages are we Building, Actually?

Wer misst, misst Mist. – German proverb

Opam takes its installation instructions from the opam metadata files that are collected in opam repositories like opam-repository. This is how the regular opam health check works, it selects (nearly all) packages, and attempts to build them.

However, only projects that already use Dune to build can use package management. This happens because, when building a project, you need to know which dependencies to build, where these dependencies get built and installed, and which paths to pass to the compiler so it can find the modules that the dependencies install. Unlike in opam, the packages don't get installed into a location containing all installed libraries (a switch), but into separate directories that will be composed together when building.

That means we need to be a bit more selective about which packages we are going to pick for testing. Picking projects that don't use Dune will fail in 100% of the cases and will not let us draw useful conclusions besides telling us that you need Dune projects to use Dune package management, which we already know. 

So, when determining which packages we want to include as our candidates, we need to filter the list of packages to ones that use Dune. The opam-health-check tool expects to call a shell command to generate the list. However, the process of determining which packages count as 'are using Dune' is more complicated, since the best way to determine that would be to detect whether dune build is used in a package and whether the package depends on the dune package.

It's a bit fuzzy, but we decided to only include packages that depend on the 'dune' package. This leaves us with a few false positives (e.g. packages that don't support the most recent versions of Dune) and also some false negatives (packages that accidentally capture a 'dune' dependency through their own dependencies), so this will probably need a bit of revision in the future, but for now, it should be good enough.

What About the Rest?

There are a significant number of projects using Dune and this is far from all of them. While we can't build them directly because every build system works differently, all opam packages can be used as dependencies and should just work.

How do we know this? We run different kinds of tests before using an internal tool that is quite similar but less sophisticated than opam-health-check. In a previous run on OCaml 4.14, we tested using an opam package as a dependency, attempting to build a project, and then checking the results. For that test, we selected 2505 opam packages (since they were compatible with 4.14, opam install could find a solution) and ran it over a few days. Ultimately, we only had 36 failures; thus, our success rate was a whopping 98%! This means that users can safely start using Dune for package management in their projects as the overwhelming majority of dependencies are compatible.

What is Building a Package, Really?

The biggest challenge is that much of the package metadata in the source archives is incorrect. As a result, dune pkg lock almost certainly picks invalid versions of dependencies. Why is that?

Dependencies Galore

Opam installs packages by inspecting the files in its own metadata repository, opam-repository. This repository is created by authors submitting their packages on release, and from there on, it is maintained by the opam-repository maintainers. They will make sure to add dependencies that have been accidentally left out or adjust when new, incompatible versions of dependencies get published. Older package definitions will be updated to include upper version constraints.

However, if we check out a repository via git or download the source archive and try to build it with Dune, we don't have all these updates. Without them, many packages will fail to build (be it with opam or Dune).

These issues can often be fixed very easily by the author of the package, and having Dune fail to build packages due to invalid dependencies is very disappointing. If the dependencies were to be fixed, the project would either work just fine with Dune package management (success, hooray!) or at least fail with a more interesting error. Marking it as a dependency failure does us a disservice by hiding potential errors.

Our hack to test for Dune package management compatibility rather than accurate dependency declarations was to replace the dependencies from the source archive with information from opam-repository. This was a two-step process:

1. Overwriting the opam files with the opam files from opam-repository.
2. Removing the dependency information from dune-project because Dune prioritises the information in this file by default.

Step two had an additional challenge as the dune-project file is in S-expression syntax, but the usual helpful processing tools like jq do not support S-expressions. So, we used Jane Street's sexp tool to do the processing, along with a generous helping of common Unix shell tools.

This is not to say that users should be migrating their dependency specifications out of dune-project (they shouldn't), but for our automated processing it was easier to take the updated opam files and use them as-is, instead of migrating them back into the dune-project syntax.

What is a Package, Actually?

When opam builds a package that uses Dune, it usually calls dune build -p <package-name>, which makes Dune ignore everything in the source repository that is not attached to the package name. However, it doesn't work for the health check, as you want all projects in the source archive to be built, not just the current one that is to be tested. But you also don't want to build every package from the source archive, as that might introduce additional dependencies and unrelated failures. Likewise, you don't want to build code that is not part of any package (e.g. examples, benchmark, utilities).

In the end, we solve it by determining the internal dependencies of the project to be built and then collecting these dependencies. We start the build by calling dune build --only-packages <packages-discovered> to restrict the build to only these packages.

Ok, Ok, but Show Me the Results!

The output of these runs is published on dune.check.ci.dev, where we build the candidate packages on Linux amd64 using the Dune developer preview binaries. We chose this platform because it will give us the biggest set of candidates since most packages are developed on systems similar to it. On the website, you can see all the packages we selected and the result of the build. At the time of writing we have selected 2243 packages to build and 1866 have completed the build successfully, which means that, at the time of writing, we have an 83% success rate in building projects directly! For the remaining 377 packages, the failures can be seen when clicking the entries since opam-health-check keeps logs of all the builds. It is our main tool to determine which issues to tackle next. So as we go forward we expect the success rate to rise to match opam as closely as possible!

Where Do We Go From Here?

Now that we have opam-health-check running and reporting build successes and failures, we can look into the build issues that it has revealed. A lot of them were small stumbling blocks which could have nevertheless been blockers to adoption:

• The potentially simplest issue arose from the Dune not supporting packages distributed in ZIP archives. Due to OCaml's strong origins on Unix, most packages are distributed as compressed tarballs (often gzip or bzip2 compressed). However, especially on Windows, the ZIP format is more popular and is also supported in opam. In #11511, we added Dune support for uncompressing ZIP files. We usually call programs to decompress the data to avoid shipping implementations of compression algorithms. However, to use these programs, they need to be available, and what is available depends on the platform. The simplest command to call is unzip from the Info-ZIP project. Still, on some platforms, the tar command also supports decompressing ZIP files as if they were tarballs, so we're trying to use whatever the user might have available.
• When pinning a package, we assume it uses Dune. This works most of the time because a significant number of packages use Dune to build, but if a package does not, we will have to build and install it using the commands that it declares in its opam file. #11513 does just that. It extracts the commands when pinned and uses them when the pinned package needs to be built.
• A somewhat obscure semantic of the way dependencies and conflicts are represented in opam files is that packages which are dependencies are implicitly conjunctions (depending on foo, bar means depending on foo AND bar); however, for conflicts, they are implicitly disjunctions (conflicting with foo and bar means to conflict with foo OR bar). This makes a lot of sense intuitively but is easily forgotten. Dune used to accept a conflict only if all packages were conflicting, and this behaviour flew under the radar for a long time because conflicts are rare. Most of the time, the conflict is only a single package, in which case it doesn't make a difference. This was fixed in  #11515, which also simplified the code.
• When solving a project's dependencies, the solver has to go through all of them and find a solution that satisfies all constraints, or it will display an error. These constraints are usually declared in your dune-project or .opam files, but when using Dune package management, there is an additional constraint: the solution needs to be buildable with the currently running version of Dune. Unfortunately, in such a case, the solver would crash. In #11554, we solved the issue to some degree: instead of crashing, the solver will display an error message, which will hopefully make it clearer why it can't find a solution.
• Opam has a little-known but very useful feature when declaring package dependencies. Instead of depending on a specific version, the user can use the current version of the package as a variable. This allows projects that consist of multiple packages to depend on each other without having to update all dependencies on every release (an example of this is ocaml-zmq, which comes with async and lwt variants which depend on a common core). However, these constraints don't matter much when building the packages, so we always set the version to dev. Unfortunately, this can cause subtle issues where no solution can be found, so in  #11517, the code was changed to attempt to read the version fields to populate the variable with the value the user declared.
• At the moment, Dune handles the compiler in a special way. When attempting to build the compiler, instead of building it in the project, it will build it in a separate location in the user's home directory. This is due to the fact that the compiler can't be moved to a different location at the moment (work is underway to improve the situation - that effort is called "relocatable OCaml"). How OCaml 5.3.0 is packaged in opam-repository changed and introduced a new transitive dependency for the compiler. Thus, the code would not be able to properly detect which opam package is the compiler. This was fixed in #11310 by computing the dependency cone of all possible compiler packages that are currently used to detect which package contains the compiler.
• Opam has a way to mark a package as 'do not pick this package unless requested explicitly' - avoid-version. This is, for example, used to mark beta versions of packages that can be installed manually but should not be automatically picked. The solver in Dune does not have such a feature, so originally, Dune sorted these packages to the end of the candidate list, but it would not match the semantics of opam. Dune would then interpret them as forbidden dependencies. However, some older packages failed to build without access to these dependencies, so #11494 was implemented where, instead of failing, the solver tries to minimise the number of dependencies picked that have the avoid-version flag.
• Findlib, the tool whose package specification format is prevalent in the OCaml ecosystem and is also used by Dune, has a feature where parts of packages are installed in subdirectories. These subdirectories can also be optional when certain package features are enabled or disabled during building. It is a rare feature, but some real-world packages use it. Unfortunately, Dune would always assume that these directories existed if they were declared and try to read their contents. But if the directory does not exist (e.g. the feature is disabled), this would lead to a crash. The fix in #11569 is short and shows that all bugs are shallow if enough eyes inspect the code.

Fixing these issues has gotten us to an (at the time of writing) 83% success rate in building projects according to opam-heath-check. That's a pretty good result and makes us confident that the package management feature is on the right track.

The issues above, as well as future issues related to package coverage and their status, are collected in a tracking issue on the Dune bug tracker.

How You Can Help

If you want to take part in improving our OCaml ecosystem to have a simple, one-stop-shop for building and installing packages check out the nightly developer preview and try it with your projects. The team is looking for feedback on how they can improve Dune package management, so please share your thoughts on Discuss, and report any issues on GitHub!

Stay in touch with us on Bluesky, Mastodon, Threads, and LinkedIn. We look forward to hearing from you!

The natural next step, considering the growing need for agile multi-purpose satellites and the suitability of OCaml-based solutions, is to put OCaml to work in space! Following up on our sister company Parsimoni’s SpaceOS project, there has been an exciting development on this front. Parsimoni has partnered with DPhi Space and put SpaceOS software aboard their Clustergate ride-sharing platform for hosted payloads. OCaml launched into space aboard Transporter-13 on the 15th of March.

The Clustergate Platform

Clustergate is a payload platform developed by DPhi Space and deployed on a host satellite. The goal of this platform is to offer the power and computing capabilities of a larger satellite to cube-sat-sized payloads at a lower cost, where customers only pay for what they need. Making satellite deployment more accessible and agile will change the future of space innovation, incentivising new actors and services.

Parsimoni is changing the way that satellite software is designed, centring on the security, efficiency, and scalability of satellite payload management. By utilising unikernel technology written in OCaml, SpaceOS can host multiple applications with a reduced attack surface, safe from a multitude of common security vulnerabilities, without the overhead of typical virtual machines.

What’s on the Satellite?

DPhi Space has embedded its own Clustergate computer on Transporter-13, and the team behind SpaceOS has onboarded OCaml 5 software on the satellite. As part of a larger ‘rideshare’ mission, DPhi Space’s computer hosts a variety of software and hardware from different partners.

So, what OCaml code is on the satellite? All in all, the team have included a simple version of Petrel to manage unikernels, the necessary ‘glue code’ to give unikernels access to orbital data, the basic functionality needed to manage data transfers and send commands, plus a ‘hello world’ unikernel. Petrel is an experimental unikernel manager and orchestrator (written in OCaml 5 with Eio concurrency) based on Albatross by Robur. Instead of hardware virtualisation (which is not available on this flight), our team uses the solo5-spt backend of MirageOS as the unikernel runtime, which leverages Linux’s seccomp feature to isolate the software payloads.

During the mission, they will test whether the system works by sending new MirageOS unikernels using the data onboard. Parsimoni expects to start testing the software onboard in May. The goal is to show SpaceOS in action, sending and receiving interesting data and deploying self-contained applications on a limited bandwidth. They will start with the hello world and move on to more complex tasks utilising orbital data.

Until Next Time

You can watch the launch of Transporter-13 to see the moment when OCaml travels through the atmosphere! If you want to discuss the opportunities that SpaceOS offers for deploying specialised and secure applications that use limited resources efficiently, you can contact us or Parsimoni to find out more.

Connect with Tarides online on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects.

When I realised that FOSDEM was still taking proposals for Birds-of-a-Feather (BOF) sessions, I submitted a proposal to organise a Functional Languages BOF session around the idea of gathering the Functional Programming community to showcase projects and elegant solutions to real world programming problems.

I was surprised and happy when the proposal was accepted by the FOSDEM organisers late Saturday night, during the conference, leaving just enough time to prepare and call in the Functional Programming community at FOSDEM for our Sunday afternoon session. Thankfully, FOSDEM runs a Matrix chat server for the conference, so it was simple to announce this last-minute addition to the conference schedule.

Arriving at the Friendly Functional Languages BOF Room

On Sunday afternoon, almost 50 functional programming enthusiasts filled room H.3242 for our "Friendly Functional Languages Show and Tell" session. The turnout exceeded our expectations by far and represented a diverse cross-section of the functional programming community. We had developers from various language communities including OCaml, Gleam, Elm, Elixir, Haskell, and several others.

Show and Tell

The format was intentionally casual – a space for practitioners to share real-world code they're proud of and discuss practical applications of functional programming principles.

During these open sessions, participants presented programming techniques, API choices, interaction with an IDE, concepts inherent to programming and, of course, their projects! We saw parser combinators (live-coded in Haskell), the use of a monad to implement ‘undo’ functionality over composable operations (which Paul-Elliot presented in the context of his personal OCaml project Slipshow), compile-time SQL query generation in Gleam, effect abstraction in Haskell, and a turn-based videogame with a frontend built on Elm. It was clear that people are active and that they have a lot to say in 5 minutes!

I found it very interesting to see people demonstrate how functional programming can elegantly solve complex problems.

Looking Forward: Organising an FP Dev Room 2026

Perhaps the most exciting outcome of our BOF session was the discussion we initiated towards the end of the session about establishing a Functional Programming dev room at FOSDEM 2026. Since only the most mainstream programming languages have a realistic chance at having their application for a dev room at FOSDEM accepted, many attendees expressed interest in creating a space for FP languages next year, and several volunteered to help organise it.

We did a quick brainstorming session on what kinds of sessions an FP dev room at FOSDEM should host in order to achieve more visibility of functional programming in the Open Source scene. A major theme that emerged is that we need to show what can be and is being built with these languages in production environments. For example, OCaml is used by Ahrefs to build their leading SEO platform and by Jane Street to power their trading operations, which handled $17 trillion worth of securities trades in 2020 using a codebase of 65 million lines of OCaml. Erlang is another example, as it is used to power WhatsApp, supporting billions of active users.

Engaging the Open Source Community

As a Developer Advocate for OCaml at Tarides, I was happy to connect with developers and organisations within the Open Source software ecosystem. Several developers curious about OCaml volunteered to participate in recorded user testing sessions for the OCaml tooling we're developing at Tarides.

These kinds of direct interactions are invaluable for understanding how developers approach OCaml, what challenges they face, and what opportunities exist to make the language more accessible and powerful. People's willingness to contribute time to help improve the ecosystem speaks volumes about the collaborative spirit of the open-source community.

Final Thoughts

Organising a BOF session at FOSDEM was a somewhat spontaneous decision, but it proved to be an excellent opportunity to bring together functional programming enthusiasts in an informal setting.

I'm looking forward to seeing how the seeds planted during this session grow into a more established functional programming presence at FOSDEM 2026. If you're interested in joining the organising team for next year's prospective FP dev room at FOSDEM, feel free to reach out to sabine@tarides.com.

Thank you to everyone who attended and made the session such a success!

Connect with Tarides online on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects.

Sabine Schmaltz is a Developer Advocate at Tarides, focusing on OCaml community engagement and developer experience.

The most exciting part of this project is that we will develop tools to automate parts of the transition and document how we achieve it, which will be great resources for the wider OCaml community. This work is made possible thanks to a grant from the NLnet Foundation, which funds research and development projects furthering internet technologies and the open internet, and the NGI Zero Core fund of the European commission. This post will give you an overview of the tools, the goals of the project, and some of the methods we will use.

Why Ocsigen and Why Eio?

Ocsigen is a web and mobile framework composed of several projects and libraries including Eliom, Js_of_ocaml, Ocsigen Server, and Lwt. Ocsigen lets you build a variety of applications, from simple server-side web sites to complex client-server web and mobile apps. It is built using OCaml and benefits from its strong type system to reduce development time, simplify refactoring, and reduce the likelihood of bugs. It is one of the biggest open source projects in OCaml, and is used commercially to run the BeSport app.

Choosing a large and established project like Ocsigen will give the community a well-documented proof-of-concept of what the transition between different concurrency models looks like.

Lwt, a monadic-style concurrent programming library for OCaml, is developed as part of the Ocsigen umbrella. It has served as a way of managing I/O operations using promises for many different OCaml projects, and it is significant that Lwt’s own inventor and biggest user Ocsigen is now making the switch to effects.

The monadic style has several advantages over more traditional concurrency models (like preemptive threading or interaction loops) including fewer data races and straightforward writing, but also comes with drawbacks in comparison to direct-style effects-based concurrency. Namely, creating an abundance of heap allocations and introducing the function colouring problem to the user’s programs. With direct-style concurrency it is possible to write code in a natural, direct, style as opposed to callback-style with considerations for which code is concurrent and which is not. For more information about concurrency using effect handlers, check out our blog post on Eio.

Switching to effects is also a first step towards enabling the use of multicore features, which is not fully possible with Lwt.

Goals of the Project

The aim of the project is to create a straight-forward path for developers who want to transition projects to OCaml 5 and its new direct-style concurrency libraries. To this end, the team will develop tools for rewriting monadic syntax into direct style, rewrite interfaces, and automate the use of libraries like picos-lwt and eio-lwt. They will also develop heuristics for detecting places in the code where manual intervention is required, simplifying the developer workflow.

Put simply, we are going to:

• Automate the aspects of transforming monadic style concurrency to direct style concurrency that we can,
• Make manual intervention as smooth as possible,
• Document the process so that it is easy to replicate, troubleshoot, and adapt for other projects.

Making effects-based concurrency easier to adopt means that more OCaml developers can potentially take advantage of its benefits. Speaking from experience, Simon Grondin summed up his experiments with Eio in our interview with him from 2024:

Eio helped me reason about my code, and I discovered bugs and problems because of how much Eio had cleaned up the code. I uncovered hidden bugs in every program I converted from Lwt to Eio. Every single one also ended up being faster, not because Eio itself was faster (it was as fast as Lwt), but because of the optimisations I could now afford to make, thanks to the reduced complexity.

This project will enable more people to try Eio and see if their experience matches Simon’s, with the potential to significantly improve their workflow!

Challenges and Methods

One of the biggest challenges facing this work is the way that in an effect-based library there is an explicit fork feature to create a new thread, whereas forking is implicit with Lwt which makes it hard to detect. This fact alone is the reason why the team won’t be able to write a fully automated conversion tool.

Another challenge for the team is to make sure that they stay as neutral as possible in their approach to the effect library's design, in order to be able to make changes later or provide multiple alternatives.

Finally, they will strive to maintain backward compatibility to the greatest extent, by using Lwt-effect bridges to enable intercompatibility for existing applications without forcing them to switch immediately.

Until Next Time

Keep your eye on our blog and OCaml Discuss for more updates on this project and the tools that emerge from it.

Connect with Tarides online on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects.

One of these features is memory profiling, which, after much theoretical consideration, has been successfully adapted to OCaml 5. Memory profiling is an important tool for developers who want to optimise their programs, and our post today delves into OCaml 5’s statistical memory profiler, statmemprof, and its now multicore-compatible design. Let’s explore the journey to its return!

What is a Memory Profiler?

Developers use memory profilers to understand how their programs use memory. Whether they think it’s using too much, is behaving suspiciously, or want to analyse it for comparison’s sake, attaching a memory profiler lets them see how their program allocates memory and keep track of it when it runs. It sounds straightforward, but this is where the challenges begin!

One of the first hurdles to clear is the sheer volume of allocated memory. Many programs, and in fact, many of the programs that are likely to be interesting from a memory perspective, allocate terabytes of memory over their run time. Running a memory profiler that monitors them all would significantly slow down the entire system. OCaml used to have a memory profiler that monitored all allocations (see spacetime), but it was removed because it was too resource-expensive.

The solution to this first conundrum is to use a statistical memory profiler (the ‘stat’ in Statmemprof). A statistical memory profiler records a random sample of memory allocations in the program. This method still allows users to find allocations that stand out. Large allocations of memory tend to be more noteworthy, and consequently, if you have a program that allocates small and large pieces of memory, you want the random sampler to sample the bigger ones more often.

Statmemprof was added to 4.11 while Multicore OCaml was in development. Hence, the original implementation of Statmemprof did not have to worry about how memory profiling should work with multiple domains. With the arrival of OCaml 5 with its multicore support, this question had to be addressed.

How Statmemprof Works With OCaml 5

Memory Allocation in OCaml

There are a few things one needs to wrap one’s head around to understand how statmemprof does its magic. This includes the way OCaml allocates memory with an inline pointer-bump allocator. If you are already familiar with memory allocation in the minor and major heap, jump ahead to the next section!

OCaml needs to be able to allocate millions of objects a second and, therefore, needs very efficient memory allocation. Most programming languages call a function in the language library (such as malloc) that determines which memory to allocate. This process is too slow to work well in OCaml and for many other garbage collected languages such as Haskell, which also use bump-pointer allocators.

In OCaml, part of the total memory available is reserved in what is known as the minor heap. In the minor heap, an allocation register points to the lowest address of allocated memory or to the boundary between what is allocated and free. Say a new object needs 32 bytes of memory: the system subtracts 32 bytes from where the allocation register is pointing and this space is used for the new object. When the minor heap’s garbage collector (GC) runs, it checks which objects can be reclaimed and which need to be kept. Surviving objects are promoted to the major heap, and the allocator register is reset to the start of the minor heap since it is now empty.

The minor heap has a ‘limit’, most commonly set to where the heap’s space ends, that, when reached, triggers a jump into the runtime system. The runtime can then take one of several actions, including garbage collection. This design makes memory allocation in OCaml very fast. Crucially for our topic today, this limit can be used to trigger a number of important events. Signal handling, for example, is achieved by tripping the limit in the minor heap to get into the runtime, which then runs the signal handlers. The runtime decides what actions to take and where to set the limit in the minor heap, allowing it to perform many different behaviours.

OCaml can also bypass the minor heap and allocate objects directly in the major heap. This is useful for very large objects, which tend to live longer and survive the minor heap’s GC anyway. That's a topic for another time.

With this basic overview of how OCaml allocates memory in our back pocket, let’s look at how statmemprof profiles memory in this system.

Statistical Memory Profiling in OCaml

The key to how statmemprof profiles memory lies in how the ‘statistical’ aspect is defined. To sample only a subset of memory allocations we need to define a workflow by which we get a random selection of samples. Since it only profiles every n number of allocations the user can leave the profiler running in the background without introducing significant overhead.

So how does it work? We need to generate a number for both the minor and major heap to help us select the sample we want to profile. We need the number to be random, meaning that every number has an equal probability of being generated. Statmemprof achieves this through statistical sampling using a so-called Bernoulli trial, meaning that it samples every word of memory allocation with the same probability.

Say the event we’re interested in is the allocation of a single word of memory to the minor heap. We have a parameter called ‘lambda’ for any such event, which represents the likelihood that statmemprof will sample that particular event. The random number we get, called a geometric random variable, stands for how many Bernoulli trials for some given lambda (or likelihood). You can also think of it as how long do we wait (how many events happen) before we sample one event.

This choice of distributions is driven by the sampling mechanics in each heap. For the minor heap, we need to know "when is the next sample due?" which is naturally modeled by a geometric distribution - it tells us how many trials (allocations) until we hit our first success (sample). For the major heap, since we're dealing with larger blocks of memory, we need to know "how many samples should we take in N words?" This is naturally modeled by a binomial distribution, as it represents the number of successes (samples) in a fixed number of trials (N words). The geometric distribution is also computationally efficient for triggering the GC mechanism at the right time, while the binomial distribution provides a more systematic way to sample larger memory blocks.

Now, let's imagine we get a random number, say 137. That number is subtracted from the allocation register in the minor heap, and the limit is set there. When the limit is reached, we go into the runtime, and the action we take is to take a memory profile sample. Statmemprof then generates a new number, and the process repeats. The process is the same for the major heap, but we use a binomial random variable instead of a geometric one.

The benefit of statistical memory profiling is that smaller-sized objects in the minor and major heaps are less likely to be sampled since they don’t take up as much space as larger objects. This is good because the larger objects tend to be more interesting from a memory profiling perspective.

What Happens When Statmemprof Samples an Object?

Statmemprof was designed to be a flexible mechanism that gives the programmer a lot of choice. There is no hardwired action set up for when statmemprof samples an allocation. Instead, there are a number of actions to choose from left open for users to configure. They include determining the size of the object, whether it came from the minor or major heap, and what the program was doing at the time of the object’s allocation.

When statmemprof samples an allocation it executes a callback (a construct that essentially works like a function) which is provided with details about the allocation and a backtrace. A backtrace refers to the sequence of functions that called a particular function. Backtraces are used to trace backwards from the function that triggered the allocation to the functions that called it, and so on, until it reaches the entry point of the program. What this means for statmemprof is that the API provides enough details for tools like memtrace to generate visual representations of memory use for the user's programs.

There are five different kinds of events that can trigger the callback:

1. alloc_minor: an object is allocated to the minor heap
2. alloc_major: an object is allocated to the major heap
3. promote: an object survives garbage collection and is moved to the major heap
4. dealloc_minor: an object does not survive garbage collection and is freed from the minor heap
5. dealloc_major: an object does not survive garbage collection and is freed from the major heap

So, the hypothetical lifecycle of an object could be as follows: it gets stored in the minor heap with alloc_minor. The limit is tripped in the minor heap, and the garbage collector runs. The object survives garbage collection and is moved to the major heap with promote. The garbage collector runs in the major heap, and if the object is not needed anymore, it gets freed with dealloc_major. As an object's lifecycle progresses, statmemprof will execute a callback for each event and a complete picture of it can be built up. Statmemprof is designed to be flexible and configurable, and, for example, users can choose to set the profiler to retain callback information or opt to discard it.

Memtrace

For many users, delving into the code to configure statmemprof would add an undesirable level of complexity to their workflow. The solution is to use tools like Memtrace, a profiling library that uses the statmemprof interface. By building on the statmemprof functionality, these tools enable users to profile memory in the way they want to without having to worry about the specifics of how statmemprof works. Memtrace can accumulate the allocations and callstacks from the program to get a picture of which code locations are responsible for triggering allocations. (Note that, as of writing, the 5.3 compatible version of Memtrace has yet to be released by Jane Street, but work is underway).

Memtrace was created at Jane Street to help them pinpoint memory issues like space leaks. It uses the callback API implemented by statmemprof to record allocation events in the binary format Common Trace Format (CTF). Memtrace also comes with a viewer, a helpful tool that lets developers visualise their programs and see how memory is allocated.

Generating a trace is straightforward, and Luke Maurer from Jane Street outlines the process in a great blog post on their website, and, if you want to learn more about the design of Memtrace, check out this excellent guide.

This is just one example of how restoring statmemprof support brings powerful options to users of OCaml 5. Its features support the creation and implementation of tools that let users manage and understand how their programs use memory in new and detailed ways.

Considerations for Multiple Domains

So how do multiple domains affect the design of a memory profiler? The choices we made reflect our preference after weighing our options, and not necessarily the only 'right' way to approach the problem. Below are some examples of the design choices we made while bringing memory profiling to multiple domains in OCaml:

• Let’s say you have two domains running at the same time doing different jobs separately, then one domain starts profiling its memory allocations. Should memory allocated by the other domain be sampled? For us, the answer was no. Behaviour in separate domains should be treated independently of each other.
• Say you are in one domain and you start profiling, then, from this domain, you spawn another. Should the allocations in the new domain be profiled? We chose to answer 'yes' to this, since the new domain was created to achieve the work of the original domain.
• Should call-backs keep running after the profiler has called stop? In OCaml 4, after stop was called statmemprof would essentially throw away all of its sampled information. In OCaml 5 the user can determine whether to ask the profiler to stop sampling, where statmemprof stops sampling new allocations but keeps the information, or stop and discard where the profiler discards all the information held for that profile. This wasn’t a relevant feature for OCaml 4 since a terminated domain meant the program had ended and statmemprof could just disregard that information. With OCaml 5, longer running memory profiling is more likely, and we need to be able to distinguish between the two stop calls.
• For statmemprof, one domain can start a ‘profile’ by calling the start function of statmemprof and sets up all the callbacks and sampling separately from all other domains. In theory, you could apply entirely different profiling tools, like memtrace, in different domains in the same program.
• Let’s say you run a program on multiple domains and run a profile on one domain which allocates some objects, samples them, and runs the allocated callbacks. Let’s then suppose that that domain terminates but the profile keeps running (say if another domain is running the same profile) and an allocation callback is promoted in the GC and continues its lifecycle. It is generally the rule that callbacks should be run by the domain that allocated the object, but if that original domain has terminated the callback may be run by a different domain because the object might still be alive on the major heap. When the object is freed and statmemprof would need to run a deallocation callback, it can also run that callback from a different domain if the original domain has been terminated.
• Lastly, a lot of work went into synchronisation and ensuring that no domain was ever waiting for statmemprof before being able to continue its jobs. Statmemprof only uses one lock to enforce synchronisation, which occurs when a domain terminates while statmemprof is still running. Its data is put on the orphans list which is protected by a lock. Any other domain can then adopt this data.

These were just some of the decisions that our team made to ensure the profiler worked well for programs with multiple domains, a technically complex challenge with a lot of variables to consider.

Until Next Time!

Multicore statmemprof was developed at Tarides, and we are happy to have brought the tool into the OCaml 5 era. We invite you to use the memory profiler to analyse your own programs. Please provide feedback and raise any issues in the OCaml repo and on the OCaml Discuss forum. You can also contact us directly for support with your multicore code or to get advice on how to take advantage of multicore OCaml.

Curious about how we maintain and restore features to OCaml 5? Read more of our multicore and compiler blog posts, such as compaction, compiler maintenance, and catching data races.

Connect with Tarides online on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects.

Acknowledgements

A huge thank you to Nick Barnes and Tim McGilchrist for their invaluable and extensive input on this post.

For users of Emacs, we have a brand new emacs mode for interacting with the lsp server that will make your coding experience as enjoyable as it should be. Check out the Discuss announcement at Release of ocaml-eglot 1.0.0 and the project repository at ocaml-eglot.

Without further ado, let's "unwrap" 🎁 these features for your viewing pleasure.

1. Type of Selection

This feature enhances code comprehension by allowing you to grow or shrink the selection to view updated types at different levels of granularity. You can adjust the verbosity of the type information to suit your needs, providing either a concise or detailed view. This information can be accessed conveniently through the default hover pop-up or via a dedicated output panel, making it adaptable to your workflow.

Command name: Get the Type of the Selection

• Command shortcut: Alt + T
• Grow Selection: continously press Alt + T
• Shrink Selection: Alt + Shift + T
• Add Verbosity: Alt + V

Using a dedicated Output Panel

In the settings/preferences of the ocaml platform extension, you can toggle an option to display the results of type selection in a dedicated output panel.

2. Search by Type or Polarity

Looking for functions or values that match a specific type? The Search by Type/Polarity feature let's you input a type signature, e.g., int -> string or a polarity -int +string, and then it fetches all matching functions and values across your project.

Command name: Search a value by type or polarity

Command shortcut: Alt + F

• Search by Type

• Search by Polarity

3. Construct Typed Holes

This feature let's you construct possible values for a given typed hole.

Command name: List values that can fill the selected typed-hole

Command shortcut: Alt + C

This feature also comes with a configurable option that allows it to construct values for the next typed hole automatically.

4. Jump to a specific Target

Traditional navigation, while it works, falls short when it comes to navigation in OCaml. This feature provides a seamless way to jump to specific targets which are closest to your cursor in the source code. For example, a large match construct and you could jump from one case to the next effortlesly.

Command name: List possible parent targets for jumping

Command shortcut: Alt + J

At this point, we support the following targets:

• Modules
• Functions
• Let statements
• Match statements
• Match cases (previous and next)

5. Navigate Typed Holes

This feature let's you navigate to typed holes.

Command name: List typed holes in the file for navigation

• As you move through the list with your arrow keys, the cursor jumps to the typed hole to give you a preview. When you make a selection, the cursor stays there.

• If you toggle the Navigate

• If you don't feel like jumping to a typed hole yet, just hit Esc and your cursor will portal back to it's original position.

Hope you are excited to try out these new features. It is our wish that you have a much better and smoother experience while coding OCaml in VsCode.

Please feel free to open issues if you discover a problematic behaviour:

• Issues for VSCode OCaml Platform Extension
• Issues for OCaml LSP Server
• Issues for Merlin

You can connect with Tarides on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects. We look forward to hearing from you!

We introduced you to Wasm and the benefits of bringing support for it to OCaml in our blog post on it in 2023. Since then, Wasm_of_ocaml has undergone new developments, so let’s take a look at what’s new and give you an overview of the release.

What is Wasm_of_ocaml?

Let’s start with a quick recap. Wasm_of_ocaml is a fork of the popular Js_of_ocaml compiler, translating OCaml bytecode to WebAssembly. It is web-oriented and relies on a JavaScript environment, and is designed to be an alternative to Js_of_ocaml. Since WebAssembly provides a sandboxed environment and enforces memory safety it is well-suited for security-critical applications, such as blockchain applications and programs running in the cloud. We plan to target these environments in the future.

Wasm_of_ocaml builds on the WebAssembly garbage collection extension (WasmGC), which is available by default on Chrome, Safari, and Firefox. This design means we don’t need to reimplement a garbage collector, and - as an added benefit - gives us good interoperability with JavaScript. Js_of_ocaml translates OCaml bytecode to JavaScript and is a well-liked, industrial-strength compiler for running OCaml on the web. The goal of Wasm_of_ocaml’s development is to retain the strengths of Js_of_ocaml and offer feature parity and inter-compatibility between the two compilers. You can compile your programs with Wasm_of_ocaml instead of Js_of_ocaml (you may have to make a few adjustments) and experience overall better performance.

Because of its popularity and versatility, creating an OCaml to Wasm translator has been a big priority for the team, and they continue to improve and optimise Wasm_of_ocaml over time.

What’s New?

Over the past year, much work has been done to get Wasm_of_ocaml to release readiness. Some of the changes include:

• Putting Wasm_of_ocaml into the same development repo as Js_of_ocaml: This was a natural step due to how much code the two tools share, considering Wasm_of_ocaml is a fork of Js_of_ocaml. However, the two have diverged since the former was forked away from Js_of_ocaml. To put them in the same repo required changes to bring them back in sync. This change was necessary for the first public release of Wasm_of_ocaml. These are just a subset of all the fixes and contributions, and you can check out the work in the associated PR for a more complete picture.
• Support for Wasm_of_ocaml in Dune: An important milestone on the road to the public release, this change allowed users to compile Wasm in Dune, making it much easier for existing OCaml projects to adopt the new tool. Wasm_of_ocaml support has been released in Dune 3.17.0, which you can upgrade to if you haven’t already.
• Separate compilation: Support for separate compilation enables much faster compilation when building a program. There are two PRs: the first PR brings the main update, and the second PR makes it more fine-grained and avoids having to load too many modules.
• Sourcemap support: PR #27 introduces support for source-level debugging of Wasm executables, implementing mapping between source and Wasm locations.
• Support the JS String Builtins Extension: PR #33 change enables the use of JS string builtins when available for JS engines, which allows for more efficient operations on strings.
• Minimise the use of the unsafe JS command eval: The JS command eval is known for being unsafe, and PR #24 creates an alternative workflow that minimises its use. Instead of using eval, strings can be emitted as external JavaScript fragments whenever the value of the string is known at compile time.
• Store long-lived top-level values into global variables: PR #30 introduces a change where any variable that is used a number of instructions after being defined is stored as a global variable rather than a local variable. This change improves performance and reduces the compilation time of Wasm projects.
• Updates to make Wasm_of_ocaml compatible with OCaml 5.2 and 5.3: Two PRs, #54 and #59, brought changes that made Wasm_of_ocaml compatible with OCaml: 5.2. For 5.3, PR #136 included updates to make Wasm_of_ocaml compatible with the then latest OCaml update.
• Bugfixes: Let’s round off with some bug fixes! PR #22 ensured that locals are always explicitly initialised before being used, PR #31 fixed the spec-compliance of some emitted tuple instructions, and PR #46 fixed a stack resizing bug in structural value comparison.

Benchmarks and Performance

The team has run several benchmarks with exciting results. When comparing the performance of Wasm_of_ocaml to Js_of_ocaml and the native code of OCaml’s compiler ocamlopt, the results consistently show that programs compiled with Wasm_of_ocaml are faster than ones compiled with Js_of_ocaml (but two times slower than native code). This holds true not only for microbenchmarks but on macroscopic benchmarks as well. Even more impressively, Jane Street reports that they have observed 2x-8x performance improvements using Wasm_of_ocaml compared to Js_of_ocaml.

Another aspect of performance lies in casts and bound checks. Wasm_of_ocaml uses a generic representation of values, which means that, at run time, a number of casts might be required to ensure safety. Furthermore, due to the nature of the data representation the team has chosen, a bound check is required whenever a field of value is accessed. The team found that the Wasm_of_ocaml checks take up around 10% of the execution overtime on the V8 engine and 20% on the Bonsai benchmark. The goal is to keep improving performance by reducing the amount of needless casts.

Regarding file size, Wasm_of_ocaml output code takes up more space than Js_of_ocaml, which is likely due to Wasm being a lower-level language than JavaScript. For example, Wasm_of_ocaml has to generate explicit code to allocate closures and access the environment, both implicit in JavaScript.

If you’re curious to learn more about Wasm_of_ocaml’s benchmarks and performance, Jérôme Vouillon’s talk from the ML track at ICFP 2024 goes more in-depth.

Release process, Plus a New Version of js_of_ocaml

From now on, wasm_of_ocaml and js_of_ocaml will be released jointly. For this reason, this first public release of wasm_of_ocaml is numbered 6.0.1 since it is synchronised with the release of js_of_ocaml 6.0.1.

A new and important feature of js_of_ocaml 6.0.0 is double translation, a way of making programs that use effect handlers faster. Effect handler support is realised by compiling some functions to Javascript code in continuation passing-style (CPS), which incurs a performance penalty. By passing --effects=double-translation, some functions are compiled in several versions, and the choice of which version of the function to run is made at run time. This improves performance at the cost of slightly larger Javascript bundles. More details are available on the effect handlers page of the js_of_ocaml manual.

Until Next Time

If you want to try Wasm_of_ocaml yourself, start by checking out the documentation for it in Dune and in the manual. If you and have any feedback or questions, the best way to get in touch is on Discuss or in the repo for Js_of_ocaml and Wasm_of_ocaml.

Connect with Tarides online on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects.

The main benefit of making MirageOS compatible with OCaml 5 is to make it possible to explore how to best take advantage of features unique to the latest version of the language. An early proof-of-concept thus experimented with replacing Lwt with the new concurrency library Eio. This is just one example, but it illustrates why bringing OCaml 5 support to MirageOS is a high priority for our team.

To port MirageOS to OCaml 5 we have been contributing to the port using Solo5 backends but we are also exploring the possibility to use Unikraft backends. The Solo5 port has been released recently, with the versions 1.x of the ocaml-solo5 package on opam. As a complementary effort, work has also been ongoing to make cross-compilation easier in general. This will benefit not just the MirageOS project but also any cross-compilation projects for OCaml in the future. Let’s dive into the updates!

The Solo5 Backend

The first goal was to restore support for a backend that was available to OCaml 4.14 MirageOS users: Solo5. Solo5 is popular with developers building MirageOS unikernels, and since it is currently the only fully supported base for building a freestanding Mirage application, getting Solo5 support for OCaml 5 is a big step in the right direction for OCaml users.

The PRs that introduce OCaml 5.2.1 support via the Solo5 API is the fruit of the collective effort of many developers, including, in no particular order: Pierre Alain, Fabrice Buoro, Romain Calascibetta, Christiano Haesbert, Samuel Hym, @kit-ty-kate, Hannes Mehnert, and many more helpful eyes.

So, what is Solo5? Mirage applications can run in a hypervisor, such as KVM or Xen, without a full OS, and Solo5 provides minimal services (such as telling the time, reading or writing a block on the disk or network, etc.) for running an application there. OCaml-Solo5 adds the extra libraries required to build the OCaml runtime on top of Solo5.

What has Changed?

As you can imagine, the significant features introduced in OCaml 5 depend on correspondingly large changes to its underlying design. These include assumptions of how the OS works, how the C compiler works, and how the build system is set up. All of these modifications have a direct impact on what OCaml-Solo5 must provide to build the OCaml runtime for MirageOS unikernels.

If you prefer to dive right into the code changes, I recommend that you visit the PR directly. Otherwise, let’s take a look at what’s new!

Nolibc Extensions The usage of (OS) threads has changed, as well as how memory is managed (relying in particular on mmap/munmap), some C features such as thread-local storage and C11 atomics are now required. Support for all of these must be added in a freestanding setting such as MirageOS, even if Solo5 remains monocore. Therefore, to make OCaml-Solo5 compatible with the latest release, developers have amended the nolibc library and the way the C compiler is invoked. The modifications come in the form of extensions to nolibc, and most of the nolibc extensions included in the PR are inherited from previous PRs by @kit-ty-kate, Romain Calascibetta, and Pierre Alain, including changes to pthread, mmap and TLS.

Build System To address the build system changes, the PR applies version-specific patches to sources when fetched. This replaces the previous method of modifying the OCaml build system with seds and echos. The reasoning behind this change is twofold: Firstly, all bar one of the patches have been designed to improve how the compiler build system supports cross-compilation, simplifying the maintenance of the OCaml and Solo5 compatibility for MirageOS. Secondly, making the modification system into separate patches with full explanation messages makes reviewing them easier and clarifies to the user which modified build system they rely on. A neat bonus of this restructuring is that the .opt versions of the compiler are now also built, which should result in better performance, particularly when building large unikernels.

Toolchain The update also means that the ocaml-solo5 package now installs a new {aarch64,x86_64}-solo5-ocaml-* toolchain. Creating a toolchain avoids baking the build-time directories containing nolibc and openlibm into the generated OCaml compilers. The package generates two versions of the toolchain: one with built-time directories, that is added to PATH only when the compiler builds, and the other with the final destination directories, installed in the bin directory by opam.

Beside adding compatibility with OCaml 5 for Solo5, we are also exploring alternative options. In particular, we are working on adding Unikraft support to test whether it can provide better performance.

The Unikraft Backend

Unikraft is a Unikernel Development Kit that lets users create custom unikernels with a large support for standard APIs to help port applications. It is an open-source project maintained and supported by over fifty active contributors. The main benefit of adding support for a Unikraft backend to MirageOS is to improve I/O performance in comparison to the Solo5 API.

Because of these potential benefits, adding the Unikraft backend is a high priority. Currently, there are several repositories being worked on, the most mature of which is ocaml-unikraft. The project is still in an exploratory phase, and more updates will follow when we have more to share.

Making the Build of OCaml Cross Compilers Easier

In addition to new backends, part of improving the user experience with OCaml 5 has focused on improving the compiler’s build system, in particular regarding cross compilers, which is helpful for MirageOS users since OCaml-Solo5 is really a cross compiler to the Solo5 target.

The first step to streamlining the compiler’s build system involved reducing the number of Makefiles down to one, the root Makefile. By bringing all the build logic into one place and avoiding duplication and stacking dependencies, the compiler’s build system is more consistent and easier to use. The effort is split into many PRs, both big and small, including #11243, #11248, #11268, #11420, #11675.

In addition to reducing the number of Makefiles, the build system improvements also involved improving ocamldep. It needed to be able to distinguish between source vs build trees and have support for lex and yacc input files. The effort also included breaking the dynlink library’s dependency on compilerlibs to make the build system simpler and faster (#11996).

Continuing on this work, several PRs have brought more fixes and improvements to cross-compilation, with more on the way. Currently, the largest upstreamed PRs are #13281, #13282, #13526, and #13312. If you use the new OCaml 5 capabilities in Mirage with either the Solo5 or Unikraft, you can take advantage of the simplified build systems and cross-compilation.

Try it Yourself and Stay in Touch

OCaml-Solo5 is released and you can get started simply by building MirageOS in a 5.2.1 switch, with no pinning involved. Samuel wrote a quick guide on how to get started with ocaml-solo5 in a Discuss post, and we recommend you give it a try. For Unikraft, keep an eye out for updates as work continues behind-the-scenes.

You can connect with us on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects. We look forward to hearing from you!

TL;DR – In 2024, Tarides focused on removing adoption friction with better documentation and tools; and on improving adoption via the integration with three key thriving ecosystems: multicore programming, web development, and Windows support. Updates to ocaml.org improved onboarding and documentation, while the Dune Developer Preview simplified workflows with integrated package management. Merlin added support for project-wide reference support and odoc 3, which is about to be released. OCaml 5.3 marked the first stable multicore release, and js_of_ocaml achieved up to 8x performance boosts in real-world commercial applications thanks to added support for WebAssembly. On Windows, opam 2.2 brought full compatibility and CI testing to all Tier 1 platforms on opam-repository, slowly moving community packages towards reliable and better support for Windows. Tarides’ community efforts included facilitating the creation of the first FUN OCaml conference by both sponsoring and contributing resources to organise it, hosting many local meetups, and two rounds of Outreachy internships.

Better Tools: Toward a 1-Click Installation of OCaml

Our primary effort in 2024 was to continue delivering on the OCaml Platform roadmap published last year. We focused on making it easier to get started with OCaml by removing friction in the installation and onboarding process. Our priorities were guided by the latest OCSF User Survey, direct user interviews, and feedback gathered from the OCaml community. Updates from Tarides and other OCaml Platform maintainers were regularly shared in the OCaml Platform Newsletter.

OCaml.org

OCaml.org is the main entry point for new users of OCaml. Tarides engineers are key members of the OCaml.org team. Using privacy-preserving analytics, the team tracked visitor behaviour to identify key areas for improvement. This led to a redesign of the installation page, simplifying the setup process, and a revamp of the guided tour of OCaml to better introduce the language. Both pages saw significant traffic increases compared to 2023, with the installation page recording 69k visits, the tour reaching 65k visits and a very encouraging total number of visits increasing by +33% between Q3 and Q4 2024

Efforts to improve user experience included a satisfaction survey where 75% of respondents rated their experience positively, compared to 17% for the previous version of the site. User testing sessions with 21 participants provided further actionable insights, and these findings informed updates to the platform. The redesign of OCaml.org community sections was completed using this feedback. It introduced several new features: a new Community landing page, an academic institutions page with course listings, and an industrial users showcase. The team also implemented an automated event announcement system to inform the community of ongoing activities.

Progress and updates were regularly shared through the OCaml.org newsletters, keeping the community informed about developments. Looking ahead, the team will continue refining the platform by addressing feedback, expanding resources, and monitoring impact through analytics to support both new and experienced OCaml users. Lastly, the infrastructure they build is starting to be used by other communities: Rocq just announced their brand new website, built using the same codebase as ocaml.org!

Dune as the Default Frontend of the OCaml Platform

One of the main goals of the OCaml Platform is to make it easier for users—especially newcomers—to adopt OCaml and build projects with minimal friction. A critical step toward this goal is having a single CLI to serve as the frontend for the entire OCaml development experience (codenamed Bob in the past). This year, we made significant progress in that direction with the release of the Dune Developer Preview.

Setting up an OCaml project currently requires multiple tools: opam for package management, dune for builds, and additional installations for tools like OCamlFormat or Odoc. While powerful, this fragmented workflow can make onboarding daunting for new users. The Dune Developer Preview consolidates these steps under a single CLI, making OCaml more approachable. With this preview, setting up and building a project is as simple as:

1. dune pkg lock to lock the dependencies.
2. dune build to fetch the dependencies and compile the project.

This effort is also driving broader ecosystem improvements. The current OCaml compiler relies on fixed installation paths, making it difficult to cache and reuse across environments, so it cannot be shared efficiently between projects. To address this, we are working on making the compiler relocatable (ongoing work). This change will enable compiler caching, which means faster project startup times and fewer rebuilds in CI. As part of this effort, we also maintain patches to core OCaml projects to make them relocatable – and we worked with upstream to merge (like for ocamlfind). Tarides engineers also continued to maintain Dune and other key Platform projects, ensuring stability and progress. This included organising and participating in regular development meetings (for Dune, opam, Merlin, ppxlib, etc.) to prioritise community needs and align efforts across tools like Dune and opam to avoid overlapping functionality.

The Dune Developer Preview is an iterative experiment. Early user feedback has been promising (the Preview’s NPS went from +9 in Q3 2024 to +27 in Q4 2024), and future updates will refine the experience further. We aim to ensure that experimental features in the Preview are upstreamed into stable releases once thoroughly tested. For instance, the package management feature is already in Dune 3.17. We will announce and document it more widely when we believe it is mature enough for broader adoption.

Editors

In 2024, Tarides focused on improving editor integration to lower barriers for new OCaml developers and enhance the experience for existing users. Editors are the primary way developers interact with programming languages, making seamless integration essential for adoption. With more than 73% of developers using Visual Studio Code (VS Code), VS Code is particularly important to support, especially for new developers and those transitioning to OCaml. As part of this effort, Tarides wrote and maintained the official VS Code plugin for OCaml, prioritising feature development for this editor. We also support other popular editors like Emacs and Vim—used by many Tarides engineers—on a best-effort basis. Improvements to OCaml-LSP and Merlin, both maintained by Tarides, benefit all supported editors, ensuring a consistent and productive development experience.

While several plugins for OCaml exist (OCaml and Reason IDE–128k installs, Hackwaly–90k installs), our OCaml VS Code plugin –now with over 208k downloads– is a key entry point for developers adopting OCaml in 2024. This year, we added integration with the Dune Developer Preview, allowing users to leverage Dune's package management and tooling directly from the editor. Features such as real-time diagnostics, autocompletion, and the ability to fetch dependencies and build projects without leaving VS Code simplify development and make OCaml more accessible for newcomers.

The standout update in 2024 was the addition of project-wide reference support, a long-requested feature from the OCaml community and a top priority for commercial developers. This feature allows users to locate all occurrences of a term across an entire codebase, making navigation and refactoring significantly easier—especially in large projects. Delivering this feature required coordinated updates across the ecosystem, including changes to the OCaml compiler, Merlin, OCaml LSP, Dune, and related tools. The impact is clear: faster navigation, reduced cognitive overhead, and more efficient workflows when working with complex projects.

Additional improvements included support for new Language Server Protocol features, such as signature_help and inlay_hint, which enhance code readability and provide more contextual information. These updates enabled the introduction of new commands, such as the "Destruct" command. This little-known but powerful feature automatically expands a variable into a pattern-matching expression corresponding to its inferred type, streamlining tasks that would otherwise be tedious.

Documentation

Documentation was identified as the number one pain point in the latest OCSF survey. It is a critical step in the OCaml developer journey, particularly after setting up the language and editor. Tarides prioritised improving odoc to make it easier for developers to find information, learn the language, and navigate the ecosystem effectively. High-quality documentation and tools to help developers get "unstuck" are essential to reducing friction and ensuring a smooth adoption experience.

Tarides is the primary contributor and maintainer of odoc, OCaml’s main documentation tool. In preparation for the odoc 3 release, our team introduced two significant updates. First, the odoc Search Engine was integrated, allowing developers to search directly within OCaml documentation via the Learn page. Second, the odoc Cheatsheet provides a concise reference for creating and consuming OCaml documentation. We would like to believe that these updates, deployed on ocaml.org, were the main cause of a 45% increase in package documentation usage on https://ocaml.org/pkg/ in Q4 2024!

Another area where developers often get stuck is debugging programs that don’t work as expected. Alongside reading documentation, live debuggers are crucial for understanding program issues. Tarides worked to improve native debugging for OCaml, focusing on macOS, where LLDB is the only supported debugger. Key progress included a name mangling fix to improve symbol resolution, restoring ARM64 backtraces, and introducing Python shims for code sharing between LLDB and GDB.

OCaml’s error messages remain a common pain point, particularly for syntax errors. Unlike Rust’s error index, OCaml does not (yet!) have a centralised repository of error explanations. Instead, we are focused on making error messages more self-explanatory. This requires developing new tools, such as lrgrep, a domain-specific language for analysing grammars built with Menhir. lrgrep enables concise definitions of error cases, making it possible to identify and address specific patterns in the parser more effectively. This provides a practical way to improve error messages without requiring changes to the compiler. In December 2024, @let-def successfully defended his PhD (a collaboration between Inria and Tarides) on this topic, so expect upstreaming work to start soon.

OCaml Package Ecosystem

The last piece of friction we aimed to remove in 2024 was ensuring that users wouldn’t encounter errors when installing a package from the community. This required catching issues early—before packages are accepted into opam-repository and made available to the broader ecosystem. To achieve this, Tarides has built and maintained extensive CI infrastructure, developed tools to empower contributors, and guided package authors to uphold the high quality of the OCaml package ecosystem.

In 2024, Tarides’ CI infrastructure supported the OCaml community at scale, handling approximately 20 million jobs on 68 machines covering 5 hardware architectures. This infrastructure continuously tested packages to ensure compatibility across a variety of platforms and configurations, including OCaml’s Tier 1 platforms: x86, ARM, RISC-V, s390x, and Power. It played a critical role during major events, such as new OCaml releases, by validating the ecosystem’s readiness and catching regressions before they impacted users. Additionally, this infrastructure supported daily submissions to opam-repository, enabling contributors to identify and resolve issues early, reducing downstream problems. To improve transparency and accessibility, we introduced a CI pipeline that automates configuration updates, ensuring seamless deployments and allowing external contributors to propose and apply changes independently.

In addition to maintaining the infrastructure, Tarides developed and maintained the CI framework running on top of it. A major focus in 2024 was making CI checks available as standalone CLI tools distributed via opam. These tools enable package authors to run checks locally, empowering them to catch issues before submitting their packages to opam-repository. This approach reduces reliance on central infrastructure and allows developers to work more efficiently. The CLI tools are also compatible with GitHub Actions, allowing contributors to integrate tests into their own workflows. To complement these efforts, we enhanced opam-repo-ci, which remains an essential safety net for packages entering the repository. Integration tests for linting and reverse dependencies were introduced, enabling more robust regression detection and improving the reliability of the ecosystem.

To uphold the high standards of the OCaml ecosystem, every package submission to opam-repository is reviewed and validated to ensure it meets quality criteria. This gatekeeping process minimises errors users might encounter when installing community packages, enhancing trust in the ecosystem. In 2024, Tarides continued to be actively involved in maintaining the repository, ensuring its smooth operation. We also worked to guide new package authors by updating the contributing guide and creating a detailed wiki with actionable instructions for adding and maintaining packages. These resources were announced on Discuss to reach the community and simplify the process for new contributors, improving the overall quality of submissions.

Playing Better with the Larger Ecosystem

Concurrent & Parallel Programming in OCaml

Twenty+ years after this statement, processors are multicore by default, and OCaml has adapted to this reality. Thanks to the combined efforts of the OCaml Labs and Tarides team, the OCaml 5.x series introduced multicore support after a decade of research and experimentation. While this was a landmark achievement, the path to making multicore OCaml stable, performant, and user-friendly has required significant collaboration and continued work. In 2024, Tarides remained focused on meeting the needs of the broader community and commercial users.

OCaml 5.3 (released last week) was an important milestone in this journey. With companies such as Routine, Hyper, and Asemio adopting OCaml 5.x, and advanced experimentation ongoing at Jane Street, Tezos, Semgrep, and others, OCaml 5.3 is increasingly seen as the first “stable” release of the multicore series. While some performance issues remain in specific parts of the runtime, we are working closely with the community to address them in OCaml 5.4. Tarides contributed extensively to the 5.2 and 5.3 releases by directly contributing to nearly two-thirds of the merged pull requests. Since Multicore OCaml was incorporated upstream in 2023, we have been continuously involved in the compiler and language evolution in collaboration with Inria and the broader OCaml ecosystem.

Developing correct concurrent and parallel software is inherently challenging, and this applies as much to the runtime as to applications built on it. In 2024, we focused on advanced testing tools to help identify and address subtle issues in OCaml’s runtime and libraries. The property-based test suite reached maturity this year, uncovering over 40 critical issues, with 28 resolved by Tarides engineers. Trusted to detect subtle bugs, such as issues with orphaned ephemerons, the suite has become an integral part of OCaml’s development workflow. Importantly, it is accessible to contributors without deep expertise in multicore programming, ensuring any changes in the compiler or the runtime do not introduce subtle concurrency bugs.

Another critical effort was extending ThreadSanitizer (TSAN) support to most Tier 1 platforms and applying it extensively to find and fix data races in the runtime. This work has improved the safety and reliability of OCaml’s multicore features and is now part of the standard testing process, further ensuring the robustness of the runtime.

Beyond testing, we also worked to enhance library support for multicore programming. The release of the Saturn library introduced lock-free data structures tailored for OCaml 5.x. To validate these structures, we developed DSCheck, a static analyser for verifying lock-free algorithms. These tools, along with Saturn itself, provide developers with reliable building blocks for scalable multicore applications.

Another promising development in 2024 was the introduction of the Picos framework. Picos aims to provide a low-level foundation for concurrency, simplifying interoperability between libraries like Eio, Moonpool, Miou, Riot, Affect, etc. Picos offers a simple, unopinionated, and safe abstraction layer for concurrency. We believe it can potentially standardise concurrency patterns in OCaml, but we are not there yet. Discussions are underway to integrate parts of Picos into higher-level libraries and, eventually, the standard library. We still have a long way to go, and getting feedback from people who actively tried it in production settings would be very helpful!

Web

Web development remains one of the most visible and impactful domains for programming languages; JavaScript, HTML, and CSS are the most popular technologies in 2024. For OCaml to grow, it must integrate well with this ecosystem. Fortunately, the OCaml community has already built a solid foundation for web development!

On the frontend side, in 2024, Tarides focused on strengthening key tools like js_of_ocaml by expanding its support for WebAssembly (Wasm). js_of_ocaml (JSOO) has long been the backbone of OCaml’s web ecosystem, enabling developers to compile OCaml bytecode into JavaScript. This year, we merged Wasm support back into JSOO, unifying the toolchain and simplifying adoption for developers. The performance gain of Wasm has been very impressive so far: CPU-intensive applications in commercial settings have seen 2x to 8x speedups using Wasm compared to traditional JSOO. We also worked on better support for effect handlers in js_of_ocaml to ensure applications built with OCaml 5 can run as fast in the browser as they used to with OCaml 4.

On the backend side, Tarides maintained and contributed to Dream, a lightweight and flexible web framework. Dream powers projects like our own website and the MirageOS website, where we maintain a fork to make Dream and MirageOS work well together. Additionally, in 2024, we enhanced cohttp, adding proxy support to address modern HTTP requirements.

While Tarides focused on JSOO, wasm_of_ocaml, Dream, and Cohttp, the broader community made significant strides elsewhere. Tools like Melange offer an alternative for compiling OCaml to JavaScript, and frameworks like Ocsigen, which integrates backend and frontend programming, continue to push the boundaries of what’s possible with OCaml on the web. Notably, Tarides will build on this momentum in 2025 through a grant to improve direct-style programming for Ocsigen.

Windows

Windows is the most widely used operating system, making first-class support for it critical to OCaml’s growth. In 2024, 31% of visitors to ocaml.org accessed the site from Windows, yet the platform’s support historically lagged behind Linux and macOS. This gap created barriers for both newcomers and commercial users. We saw these challenges firsthand, with Outreachy interns struggling to get started due to tooling issues, and commercial users reporting difficulties with workflow reliability and compilation speed.

To address these pain points, Tarides, in collaboration with the OCaml community, launched the Windows Working Group. A key milestone that our team contributed to was the release this year of opam 2.2, three years after its predecessor. This release made the upstream opam-repository fully compatible with Windows for the first time, removing the need for a separate repository and providing Windows developers access to the same ecosystem as Linux and macOS users. The impact has been clear: feedback on the updated installation workflow has been overwhelmingly positive, with developers reporting that it "just works." The install page for Windows is now significantly shorter and simpler!

In the OCaml 5.3 release, Tarides restored the MSVC Windows port, ensuring native compatibility and improving performance for Windows users. To further support the ecosystem, Tarides added Windows machines to the opam infrastructure, enabling automated testing for Windows compatibility on every new package submitted to opam. This has already started to improve package support, with ongoing fixes from Tarides and the community. The results are publicly visible at windows.check.ci.dev, which we run on our infrastructure, providing transparency and a way to track progress on the status of our ecosystem. While package support is not yet on par with other platforms, we believe that the foundations laid in 2024—simplified installation, improved tooling, and continuous package testing—represent a significant step forward.

Community Engagement and Outreach

In 2024, Tarides contributed to building a stronger OCaml community through events, internships, and support for foundational projects. The creation of FUN OCaml 2024 in Berlin was the first dedicated OCaml-only event for a long time (similar to how the OCaml Workshop was separated from ICFP in the past). The conference was a huge success thanks to a huge amount of effort from David Sancho and Dmitriy Kovalenko for taking on the majority of the organisational work, Sabine Schmaltz for running the conference and web infrastructure, and from others inside Tarides, including Claire Vandeberghe who worked on the website design. Over 75 participants joined for two days of talks, workshops, and hacking, and the event has already reached 5k+ views on YouTube (for more details, you can check out our dedicated blog post on FUN OCaml). Tarides also co-chaired the OCaml Workshop at ICFP 2024 in Milan, bringing together contributors from academia, industry, and open-source communities. These events brought together two different kinds of OCaml developers (with some overlap) bringing exciting new perspectives, changes, and developments to our community.

To expand local community involvement, Tarides organised OCaml hacking meetups in Manila and Chennai. To make it easier for others to host similar events, we curated a list of interesting hacking issues from past Cambridge sessions, now available on GitHub.

As part of the Outreachy program, Tarides supported two rounds of internships in 2024, with results published on Discuss and watch.ocaml.org. These internships not only provided great contributions to our ecosystem but also brought fresh insights into the challenges faced by new users. For example, interns identified key areas where documentation and tooling could be improved, directly informing future updates.

Tarides also maintained its commitment to funding critical open-source projects and maintainers. We continued funding Robur for their maintenance work on MirageOS (most of those libraries are used by many –including us– even in non-MirageOS context) and Daniel Bünzli, whose libraries like cmdliner are essential for some of our development.

Finally, Tarides extended sponsorships to non-OCaml-specific events, including JFLA, BobConf, FSTTCS, and Terminal Feud (which garnered over 100k views). These events expanded OCaml’s visibility to new audiences and contexts, introducing the language to a broader technical community that –we hope– will discover OCaml and enjoy using it as much as we do.

What’s Next?

As we begin 2025, Tarides remains committed to making OCaml a mainstream language. Our focus this year is to position OCaml as a robust choice for mission-critical applications by enhancing developer experience, ecosystem integration, and readiness for high-assurance use cases.

We aim to build on the Dune Developer Preview to further improve usability across all platforms, with a particular emphasis on Windows, to make OCaml more accessible to a broader range of developers. Simultaneously, we will ensure OCaml is ready for critical applications in industries where reliability, performance, and security are essential. Projects like SpaceOS showcase the potential of memory- and type-safe languages for safety-critical systems. Built on MirageOS and OCaml’s unique properties, SpaceOS is part of the EU-funded Orchide project and aims to set a new standard for edge computing in space. Additionally, SpaceOS is being launched in the US through our spin-off Parsimoni. However, these needs are not limited to Space: both the EU Cyber Resilience Act and the US cybersecurity initiatives highlight the growing demand for type-safe, high-assurance software to address compliance and security challenges in sensitive domains. Tarides believes that OCaml has a decisive role to play here in 2025!

I’d like to personally thank our sponsors and customers, especially Jane Street, for their unwavering support over the years, and to Dennis Dang, our single recurring GitHub sponsor. Finally, to every member of Tarides who worked so hard in 2024 to make all of this happen: thank you. I’m truly lucky to be sailing with you on this journey!

We are looking for sponsors on GitHub, are happy to collaborate on innovative projects involving OCaml or MirageOS and offer commercial services for open-source projects – including long-term support, development of new tools, or assistance with porting projects to OCaml 5 or Windows.

Motivation

OCaml 5.3 is released and comes with a restored MSVC (Microsoft Visual C/C++ compiler) port. It had been removed with the introduction of the multicore runtime in OCaml 5.0. The Restore the MSVC port of OCaml #12954 PR lists all the prerequisite changes, and the final streak of changes, required to restore support.

The OCaml 5 runtime requires C11 atomic support from the C compiler, which for MSVC was introduced in Visual Studio 2022 version 17.5 Preview 2. This early version had a few bugs during code generation and we were wondering if they would impact the OCaml runtime (fortunately, they did not). While the Microsoft team worked hard to fix the bugs, we learned about clang-cl, a driver program for Clang that attempts to be compatible with MSVC's cl.exe. It is ABI-compatible with MSVC, implements all MSVC compiler extensions, and is also a drop-in command-line replacement for cl.exe.

I wanted to try building OCaml with clang-cl, mostly because it would get us a second compiler's opinion on our code, as Clang has a different set of warnings and hints than MSVC. Seeing that clang-cl was already used to build Chrome and Firefox (1, 2), I was hoping the needs of the OCaml runtime would have already been covered and bugs fixed, and that we could adopt it seamlessly.

The OCaml 5 runtime uses the POSIX threads (pthreads) library on Unix-like systems for all its concurrency primitives. For the OCaml 5.3 branch we've chosen to use the winpthreads library, part of the MinGW-w64 project, which implements pthreads on Windows. The OCaml 5 MinGW-w64 port uses it, and we found out we could use it with MSVC too. I then submitted two patch series to winpthreads (Patches and cleanups towards MSVC support, MSVC support without GCC extensions), checking my work with MinGW-w64+GCC, MinGW-w64+clang, MSVC, and clang-cl, foreshadowing their use within the OCaml runtime. What an adventure! And thanks to the MinGW-w64 team for reviewing this work.

Using clang-cl

Clang on Unix-like systems masquerades as GCC, supports all GNU C extensions and defines the __GNUC__ macro. On Windows, it masquerades as MSVC and defines the _MSC_VER macro instead, but still supports the GNU C extensions that we can take advantage of!

The build system only required a few changes. For instance, MSVC currently defaults to C99 and needs two flags to switch to C11 and enable experimental C11 atomic support, whereas clang-cl defaults to C17. We also discussed how to improve the support of MSVC in Autoconf, which led to a few patches. We could then use clang-cl to discover new problems reported by the warnings it raised and fix them. In conjunction with this work, I raised the warning level of MSVC on the OCaml runtime C code from none to -W2.

Fortunately, most of the warnings were quite minor (see #13081 and #13243), mainly consisting of warnings for deprecated functions or implicit truncations when converting integers or floating points values of different sizes. Switching to newer compilers also allowed us to remove dead code and workarounds for older versions of the compilers.

I found out that most of the uses of compiler attributes or builtins were guarded by the __GNUC__ macro, and as such were only enabled by GCC or Clang on Unix, even though clang-cl on Windows supports them too. Compiler attributes may enable more checks and warnings from the compiler. For instance, the format attribute tags a function to be printf-like and checks the types of the list of values passed to it against the specifiers inside a format string. Compiler builtins may enable more optimisations, such as __builtin_expect. So, instead of guarding their use, we could discover whether the compiler provides them using newer macros such as __has_attribute or __has_builtin. This improved feature parity between the clang-cl port and an OCaml build using GCC or Clang.

In particular, we could detect the labels as values (also known as computed gotos) compiler extension to enable threaded code interpretation, which dramatically improves the speed of ocamlc, the OCaml bytecode interpreter. This optimisation isn't supported by MSVC and would require us to use inline assembly on x86 or write part of the bytecode interpreter in assembly on other architectures. If you're often using the bytecode interpreter, there's now a clear advantage of using clang-cl over MSVC. Another interesting optimisation uses software prefetching to speed up the GC when traversing the graph of values. We had worked on restoring it in OCaml 5 from OCaml 4 but forgot to port it to Windows!

The overall work on restoring the MSVC port of OCaml and also building it with clang-cl led to a few bug reports to Microsoft and to the LLVM project, which I hope will benefit the community. The OCaml project now has an extra set of compiler eyes that scrutinise each and every change on Windows. Windows users may now take advantage of a (compliant) C11 and C23 FOSS compiler, with a wide range of optimisations and checks available.

I'm grateful to my colleagues at Tarides for helping me with this work, the OCaml core team for reviewing it, and Jane Street for sponsoring this effort.

Try it Out and Stay in Touch!

With the release of opam 2.2 (now 2.3) supporting Windows, the restoration of the Cygwin port in OCaml 5.1, the MSVC port in OCaml 5.3, and the option to build OCaml with clang-cl, and the swarm of bug fixes that accompanied them, using OCaml on Windows has never been easier! Give it a whirl!

Connect with Tarides online on Bluesky, Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects.

This post highlights new and restored features, notable changes and user experience improvements, plus some bug fixes. There is no way that I can cover everything in this update, so I recommend that you check out the Changes document on GitHub for the full list of contributions!

MSVC

The 5.3 release restores support for the MSVC port of OCaml on Windows, marking the last remaining platform from 4.x to regain support in 5.x. This is part of a wider effort to achieve feature parity between OCaml 4.14 and OCaml 5, of which compaction is a previous example, making the transition between versions as smooth as possible. The bulk of the effort is summarised in PRs #12954 and #12909 opened by David Allsopp, Antonin Décimo, and Samuel Hym (review by Miod Vallat and Nicolás Ojeda Bär).

Since the OCaml 5 runtime uses C11 atomics, supported platforms need to be compatible with them as well. Visual Studio 2022 introduced experimental support for C11 atomics which made the MSVC port of OCaml 5 possible, but the team needed to test out the feature first. This exploratory effort led to several bug reports addressed by Microsoft, and once these were completed (alongside a lot of other work, including fixing the winpthreads library of the mingw-w64 project to that it builds with MSVC), the MSVC port was ready for public release.

As part of the project bringing MSVC back the team explored clang-cl, an alternative command line interface to Clang designed to be compatible with the MSVC compiler cl.exe. This was helpful because clang-cl has a different set of warnings and tips to MSVC, and using it effectively gave them a ‘second opinion’ on their code. The main PR for this side of the project is #13093.

Statmemprof

OCaml 4.14 had support for statistical memory profiling, a feature of the language that can sample memory allocations allowing tools like Memtrace to help users identify how their programs are using memory. The multicore update introduced significant complexity to the process which made it necessary to drop support for 5.0; but work soon commenced to restore support under our feature parity banner! In 5.3, statmemprof makes its return, now equipped with multicore capabilities.

So how does it work? Statmemprof can check the allocation of memory at some given frequency (lambda) per word or unit of data. By sampling a fraction of allocations at random, we are able to monitor programs in a language like OCaml which allocates high rates of memory. It would be far too expensive performance-wise to monitor every allocation.

The new design has a lot in common with the OCaml 4 implementation of statmemprof, but with several tricky optimisations and changes to account for the significant complication of multiple domains and threads. Delve into the details in the PRs #12923 and #11911 by Nick Barnes (external reviews by Stephen Dolan, Jacques-Henri Jourdan, and Guillaume Munch-Maccagnoni).

Deep Effect Handlers

OCaml 5.0 came with experimental support for algebraic effects, which allow users to describe computations and what effects they are expected to create. A handler essentially manages a computation by monitoring its execution and keeping track of resulting effects. This ‘management’ can be done in two ways, deeply or shallowly. A shallow effect handler monitors a computation until it either terminates or generates one effect, only handling that effect. A deep effect handler always manages a computation until it terminates and handles all of the effects performed by it.

PR #12309 (Leo White, Tom Kelly, Anil Madhavapeddy, KC Sivaramakrishnan, Xavier Leroy and Florian Angeletti, review by the same, Hugo Heuzard, and Ulysse Gérard) introduces effect syntax for deep effect handlers, rules that define the structure for writing them, compatible with the type checker and with support for pattern matching. This change aims to simplify the code needed to use deep effect handlers, improving user experience. Note that you can still use shallow effect handlers, and there is a good tutorial for using both in the correspondingly updated manual page.

Debugging Improvements

Another long-term project coming to fruition in this update are the several improvements to debugging on macOS. The platform is popular with a wide variety of OCaml users, including compiler developers, and they need good debugging workflows for their programs.

LLDB is the only supported native debugger on macOS, for both the ARM64 and x86_64 architectures. The improvements enable several new features:

• #13163 enable frame pointers on macOS x86_64 (Tim McGilchrist, review by Sébastien Hinderer and Fabrice Buoro): This PR introduces support for a common technique used by profiling tools including Linux perf, eBPF, FreeBSD, and LLDB, called stack-walking. Various performance tools use stack walking to reconstruct call graphs for programs, and frame pointers are what enable them to do so.
• #13241, #13261, #13271, add CFI_SIGNAL_FRAME to arm64 and RISC-V runtimes for the purpose of displaying backtraces correctly in GDB (Tim McGilchrist, review by Miod Vallat, Gabriel Scherer and KC Sivaramakrishnan): This change helps sync up the runtime for the arm64 architecture for macOS (and the RISC-V runtime) with the amd64 and s390x runtimes. The two additional PRs add improvements to the first.
• #13136 Compatible LLDB and GDB Python extensions (Nick Barnes): This PR replaces some old GDB macros (used to debug OCaml programs) with faster and more capable extensions, and makes those extensions available in LLDB. This is especially useful to macOS users who can’t use GDB.

OS-Based Synchronisation for Stop-the-World Sections

PR #12579 (B. Szilvasy, review by Miod Vallat, Nick Barnes, Olivier Nicole, Gabriel Scherer and Damien Doligez) improves user experience by replacing generic busy-wait synchronisation with OS-based synchronisation primitives, namely barriers and futexes. The change has significant performance benefits, especially on Windows machines, where spinning was causing long wait times. You can learn more about it in our blog post on the project.

User Experience Improvements

• #12868 Refresh HTML manual/API docs style (Yawar Amin, review by Simon Grondin, Gabriel Scherer, and Florian Angeletti): An update to the OCaml Manual which simplifies the colours, removes the gradients, and fixes the search button. It’s a nice improvement to a part of the OCaml ecosystem that is visible to users of all different backgrounds and contexts.
• #13201, #13244 (Sébastien Hinderer, review by Miod Vallat, Gabriel Scherer and Olivier Nicole), and #12904 (Olivier Nicole, suggested by Sébastien Hinderer and David Allsopp, external review by Gabriel Scherer) various improvements to TSan: These three PRs represent the continuous work being put in to bring improvements to TheadSanitizer or TSan. They include speedups and the ability for users to choose which PRs they want to run the TSan testsuite on.
• #13014 (Miod Vallat, review by Nicolás Ojeda Bär) add per function sections support to the missing compiler backends: This PR is an example of how much focus there is on ensuring that each native backend is equally supported, having features available across all Tier-1 platforms. Here, the compile-time option function–sections was re-enabled on all previously unsupported (POWER, riscv64, and s390x) native backends.
• #11996 emancipate dynlink from compilerlibs (Sébastien Hinderer and Stephen Dolan, review by Damien Doligez and Hugo Heuzard): The dynlink library used to depend on compilerlibs, having to embed a copy of compilerlibs meaning that it would be compiled twice, costing the user in time and performance. After the change, the build time and size of both dynlink.cma and dynlink.cmxawere reduced.

Miscellaneous Bug Fixes

These two bug fixes grew out of internship projects at Tarides, it's great to see how these projects can benefit the language as a whole.

• #13419 (B. Szilvasy and Nick Barnes, review by Miod Vallat, Nick Barnes, Tim McGilchrist, and Gabriel Scherer): This PR addressed resource leaks that caused memory bugs in the runtime events system.
• #13535 (Antonin Décimo, Nick Barnes, report by Nikolaus Huber and Jan Midtgaard, review by Florian Angeletti, Anil Madhavapeddy, Gabriel Scherer, and Miod Vallat): Expanded the documentation for Hashtbl.create to explain that negative values are allowed in the hash table but will be disregarded.

These bug fixes stem from discoveries made during the release cycle of the 5.3 update. Catching and fixing broken bits of code is an important but often lengthy part of the release process.

• #13138 (Gabriel Scherer, review by Nick Roberts): This PR is an old one, first opened eight years ago in 2016! Optimised pattern matching with mutable and lazy patterns was observed to result in occasions where seemingly impossible cases were taken, causing unsoundness issues. After lengthy efforts to narrow down the cause, the problem has been fixed for 5.3!
• #13519 (Sébastien Hinderer, report by William Hu, review by David Allsopp): This PR restored backward compatibility lost when renaming some items in Makefile.config.
• #13591 (Antonin Décimo, review by Nick Barnes, report by Kate Deplaix): This PR fixed a problem whereby compiling C++ code using the OCaml C API resulted in a name-mangled caml_state on Cygwin. The fix ensured that installed headers were compatible with C++ and protected the ones that were not with CAML_INTERNALS.
• #13471 (Florian Angeletti, review by Gabriel Scherer): Added a flag to define the list of keywords recognisable by the lexer, making adding future keywords to OCaml easier.
• #13520 (David Allsopp, review by Sébastien Hinderer and Miod Vallat): Fixed the compilation of native-code versions of systhreads.

What’s Next?

Work on OCaml continues! The next few months will bring more features and bug fixes to the language, with focus on big changes like the relocatable compiler, unloadable runtime, and laying the ground work for project-wide renaming and other powerful navigation and refactoring features. The OCaml changelog is the place to go to keep up with what’s new, as well as the OCaml Discuss forum.

You can connect with us on Bluesky, Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects. We look forward to hearing from you!

Testing APIs With Hidden or Uncontrolled State

In part 1, we saw how STM and Lin were useful to test stateful module interfaces, like Float.Array. The OCaml standard library and runtime however also expose modules that are stateful, but for which the state is hidden or outside the full control of a black-box testing process. We are nevertheless interested in helping ensure the correctness of such modules.

For example, Ephemerons depend on the state of OCaml's heap, meaning that a garbage collection (generally out of control of the test driver) may unsuspectingly trigger and cause changes in the test outcome. As a result, we ended up abandoning Lin tests of Ephemerons, as they would run multiple observations of the same system under test - one parallel observation and several sequential ones - most likely with different results, because of different garbage collection schedulings.

In addition, the Lin tests could cause excessive shrinking searches, in trying to find a minimal example causing Ephemeron differences between such runs. Instead we favoured replacing them with STM tests, that perform only one observation of the system under test. We furthermore experimented with inserting an explicit call to Gc.full_major in between our run of the sequential and parallel tests to increase the chance of starting the latter on a relatively stable heap basis. We encountered similar problems with tests of the Weak module and recently also in the STM tests of the Gc module.

Despite these challenges, the above tests successfully found several issues, e.g.

• racing Weak functions could in some cases produce strange values,
• certain combinations of Weak Hashset functions could cause the runtime to abort or segfault,
• the Ephemeron tests could trigger an assertion failure and abort, and
• out-of-date documentation for Gc.quick_stat and Gc.set

Cygwin Challenges

As Cygwin support was being restored up in the 5.1 release, we wanted to test this platform as well to help ensure its correctness. However, we found that a test-suite run took an excessively long time, often not completing within a 6-hour timeout! We solved this initially by splitting a test-suite run into two separate workflows and later by rephrasing it as two separate CI jobs, belonging to the same workflow. Thanks to general improvements to the OCaml 5 runtime system, we have since been able to merge this split back into just one job like the remaining platforms, and eventually also reduce the timeout for this single Cygwin workflow to 4 hours, like the remaining platforms.

Another Cygwin challenge early on was the relatively old opam.2.0.7 version it includes. This made it harder to test the various OCaml compiler versions on Cygwin (and later MinGW). As a workaround, we set up a custom opam repository with appropriate opam files for each compiler version.

Keeping Dependencies Minimal

Initially we happily used ppxlib to generate boilerplate code for QCheck generators using ppx_deriving_qcheck and show functions for the tested cmds. However, ppxlib depends on the OCaml compiler's AST which means that it occasionally breaks on the compiler's trunk development branch whenever its AST changes. As a consequence, testing trunk would halt until ppxlib was fixed again - an unfortunate situation when trying to help ensure its correctness! After helping keep a branch of ppxlib continuously working with trunk, at some point we instead decided to eliminate the test suite's ppxlib dependency. We therefore wrote the corresponding definitions by hand and by utilising a dedicated printing library Pp in qcheck-multicoretests-util. Since then, testing has not been blocked by such breakages, and the dependencies are down to just the qcheck-core package, and (transitively) dune.

Testing an increasing number of platforms such as the MinGW and Cygwin ports over the past 2-3 years has been challenging, as much of that effort predates the more recent Windows support brought by opam-2.2. As we additionally wanted to test OCaml 5's now restored MSVC port, also during its development, we ended up abandoning opam and setup-ocaml in favour of just building the tested compiler and our dependencies (qcheck-core and dune) from source in our CI workflows. As a result, we have gained a uniform CI workflow setup across platforms that also allows us to kick-off tests of feature branches, and thereby eliminate the need for the above-mentioned custom opam repository.

Testing in the Presence of Misbehaviour

Overall, when having found, investigated, reported, and sometimes also fixed an error, we would like to continue testing despite the existence of such known issues. A test-suite rerun is however likely to trigger and report the same bug again, temporarily hindering the discovery and fixing of other issues. Something similar has been observed by others, e.g. in Quviq's property-based testing of AUTOSAR for Volvo.

To be able to continue testing we can, in the simple cases, (temporarily) adjust the tested property to accept the observed (mis)behaviour. This was the case, for example, for Gc.quick_stat which would return non-zero entries for four record fields, where the documentation was out-of-date and specifying that zeros should be returned.

However we have also had to (temporarily) adjust the generator to avoid triggering a particular buggy cmd. This was the case, for example, for Gc.counters which had an independently found-and-fixed issue with improper C interfacing with the GC. Our new Gc test would nevertheless trigger it and cause a crash on 5.2.0, until we adjusted the generator to skip generating Gc.counters calls on tests of versions up to 5.2.0. Since then the fix has been included in the 5.2.1 bugfix release.

Finally, we have also (temporarily) disabled a test on a platform. This is the case, for example, for the parallel test of Dynlink which is unsafe on Windows, due to an underlying flexdll issue. Whereas the flexdll issue is now fixed in the flexdll repository, we are awaiting a new release before proceeding to re-enable the parallel test on Windows.

Crashes Take Down the Test Runner Too

Since the test-driver is running in OCaml too – and in the same process – when the SUT crashes, so does the entire QCheck test-driver process. In our experience this happens more often than not, as runtime issues tend to lead to memory corruption and typically a segmentation fault. Running the test in a forked child process may guard against a crash in the child taking down the parent process, with the caveat that OCaml 5 prevents forking child processes after the first Domain.spawn. In the spirit of functional programming, this option is available as a reusable combinator Util.fork_prop_with_timeout : int -> ('a -> bool) -> 'a -> bool in qcheck-multicoretests-util, thus allowing us to easily wrap the property of a crash-triggering QCheck test.

Despite not designed with any of the above in mind, we have arrived at a test layout where most tests are run as separate executables, which lets us identify crashes relatively easily and simultaneously lets a test-suite run continue despite encountering a crash underway.

Positive Testing, Negative Testing, and Stress Testing

While developing the STM and Lin libraries it became clear that we should guard against changes mistakenly affecting their error-finding behaviour. For this reason, we added negative tests of e.g. parallel modifications of an unprotected ref cell that are expected to fail, and then extended QCheck with a Test.make_neg function to construct a negative test. In the CI we then use this negative testing ability e.g. to test that Hashtbl is unsafe to use in parallel as mentioned in part 1 and that the generator can find a counterexample illustrating it.

Often, such a parallel negative test triggers quickly, e.g. on one of the first 10 test inputs generated – effectively stress-testing a module under parallel usage for only a very limited amount of time. We therefore added stress test properties, in the form of stress_test for Lin and stress_test_par for STM, both offering a weaker, more forgiving property that only fails on unexpected exceptions or outright crashes, thus strengthening our belief in the runtime - even under longer, continued parallel misusage.

Effectively, we have arrived at PBT variants of classical test concepts:

• positive tests - expected to hold across many random test inputs
• negative tests - expected not to hold and produce a counterexample
• stress tests - expected not to misbehave by raising an exception or crashing

False Alarms

As more and more CI target platforms have been added, we have also seen a variation in behaviour across them: Some of the above negative tests are not triggered as consistently across platforms and some of the tests take a long time or cause timeouts on some platforms.

These add noise and still require us to check whether a failure was genuine or not. We have therefore focused on reducing the noise from false positives. To better understand this effort here is a plot of CI workflow outcomes for merged PRs (361-486) spanning a period of ~1.5 years from early June 2023 to early December 2024:

First of all, this covers a period of testing a mix of OCaml versions, starting with workflows targeting 5.1 in June 2023, then 5.2, and now 5.3 and 5.4/trunk (the current compiler development version). Note that this only plots the outcome for Jan's PRs to main for a fair comparison, as these would originally trigger twice as many workflow runs (push and PR). It furthermore only includes the last run for each PR, in case there were more because of PR revisions. Each CI run may in principle trigger several errors (in different categories even!). This is a rare incident however.

Further notes:

• On 370 a ppc64 workflow was added (workflow number increase)
• 391 added framepointer workflows (workflow number increase)
• 392-393 revealed a cmi-file lookup regression that made several workflows fail, hence the spike
• There were CI and network issue around 395-398
• On 396 a first FreeBSD workflow was added (workflow number increase)
• On 398 a second FreeBSD workflow and extra opam install workflows were added (workflow number increase)
• On 420 the 2-split Cygwin workflows were merged (workflow number decrease)
• On 429 (merged before 431) we eliminated duplicate CI runs for both push and the PR
• On 438 we retired the 5.1 workflows to only run weekly (not on every PR)
• On 449 the older MSVC PR (399) adding 2 additional MSVC workflows had just been merged
• On 453 the parallel Dynlink tests were disabled on Windows (since a fix had been developed and offered on the FlexDLL repo)
• On 458 2 macOS ARM64 workflows were temporarily duplicated while moving them from multicoretests-ci to GitHub actions
• On 471 5.4.0+trunk workflows were added and removed three 5.1 multicoretests-ci workflows
• On 481 multicoretests-ci stopped running the 4 remaining 5.2 workflows
• On 482 the ten 5.2 workflows were disabled

The smaller sub-bars are harder to distinguish with the dominant OK bars in the above figure. Below we therefore zoom in and display the same plot, including only the different kinds of failures:

There are multiple competing efforts and aspects behind the amount of 'genuine' errors triggered in the above:

• First, OCaml developers have fixed a number of defects and released 5.1.0, 5.2.0, 5.2.1, and 5.3.0~beta2 over this time period
• Second, as new compiler features are added and merged they may accidentally introduce new errors
• Third, we have added tests and 'sharpened the axe' of the existing ones
• Finally, a fix of a bug in 5.1 merged into 5.2 doesn't prevent the bug from continuing to show up on 5.1 CI runs

It is clear from the plot that both genuine issues (categorised as 'genuine') and false alarms (categorised as 'other') have decreased over this period. The workflow alarms were initially dominated by false ones, then started being dominated by genuine (and 'ci') ones, and have now settled on a level with zero alarms being a common outcome. Such a noise-free test-suite signal is also central for OCaml compiler developers to utilise the test suite in their own development workflow. Here a test-suite red light should ideally signal a problem with a proposed runtime change, rather than (a) false alarms adding needless noise ("oh, never mind that red light!") and (b) true alarms adding genuine noise ("actually that's an existing unfixed issue, not your fault").

To understand the failures triggering on specific versions, here's first a plot of the outcomes for the weekly CI runs of the upcoming 5.3 release:

The one failure on October 21 happened during Cygwin installation, and was hence not related to the test suite. In comparison, weekly runs of the 5.0 workflows tend to trigger issues. This below plot covers 8 workflows in contrast to 5.3's 12 workflows, as Cygwin, frame-pointer, and MSVC byte/native workflows were added since. Since the 5.0 release is older than 5.3, we also started recording weekly workflow outcomes for it sooner:

The plot clearly indicates that issues are triggered on 5.0 workflows more often that not. Ever since starting to note the cause of such failures in late August 2024, the triggered issues are genuine, and typically include crashes due to parallel access to Buffer, pinpointing a case in Buffer.add_string that flew under the radar of the first Buffer fix included in the 5.0.0 release.

Hidden Costs

We have worked to reach zero false alarms and now generally achieve it across an array of 31 CI workflows. We apply due diligence and have thus developed a workflow of going over the CI red lights to understand and summarise the failures – both the genuine ones and the false alarms. We then keep track of them as issues in the multicoretests repo. This allows us to easily refer to them and spot trends, such as starting to see new failures or noticing that a particular failure no longer occurs.

When we spot new errors, we work to reproduce them locally to make sure the issue is genuine. If so, we then report the issue upstream to ocaml/ocaml along with a description of the required steps needed to reproduce it. When understanding the problem well enough, we also contribute with a compiler fix PR. Out of the 40 issues currently identified, Tarides engineers have filed PRs to fix 28 of these, 10 issues have been fixed by others (typically Inria or Jane Street engineers), and 2 issues remain open. Tarides have thus put in a significant effort to resolve errors.

Some Issues are Still Hard to Reproduce

Despite all our efforts to amplify problems and increase reproducibility, some issues are still hard to trigger. One such case was ocaml/ocaml#12707 in which we were able to trigger the assertion failure, albeit rarely. This one took some head-scratching until we realised the problem was caused by a small time window between reading the same atomic field twice in an assertion: di->backup_thread_msg == BT_INIT || di->backup_thread_msg == BT_TERMINATE. This was carried out in parallel with a backup-thread transitioning from BT_TERMINATE to BT_INIT by an atomic write atomic_store_release(&di->backup_thread_msg, BT_INIT), thus creating a tiny chance of neither of the conditions to be true, if the write would happen just in between the two reads. We could then manually insert a call to sleep to confirm, and develop an appropriate fix.

We are currently investigating an even rarer issue triggered by the Gc tests, that causes a rare crash on macOS running on an ARM64 processor and seems to require just the right OCaml heap conditions to trigger. In both cases, despite not triggering on every PBT run, the randomised tests have nevertheless highlighted genuine issues that would otherwise only show up even more rarely on the ocaml/ocaml test suite or – worse – to end users of OCaml.

Lowering the Barrier to Entry for multicoretests for Compiler Engineers

To offer compiler engineers the ability to run the test suite easily, we have made it possible to do so by labeling a PR with a run-multicoretests tag. Recently a PR to improve major GC performance with mark-delay kicked off using the run-multicoretests tag and is already making good use of the test suite as it detected an issue with the GC marking of Ephemerons. With the test suite finding an issue in another GC improving PR, we are confident in the value addition that the test suite brings to compiler developer in helping quality-assure runtime-related PRs.

Usage Outside multicoretests

The usage of STM has spread outside of multicoretests. In Saturn, a library of lock-free data structures for OCaml 5, both existing and new data structures come with STM tests to help ensure their correctness. This started in connection with moving experimental tests of ws_deque out of the multicoretests suite and has continued since. In Picos, a library for composing effect-based schedulers, STM tests also help ensure correctness of its underlying data structures.

For the Gospel specification language for OCaml, Tarides has worked to develop Ortac-QCheck-STM, as a plugin for Ortac. This is a tool to extract sequential STM tests from a Gospel specification, thereby putting the strength of PBT in the hands of OCaml developers willing to annotate their interfaces with Gospel specifications. This effort is paying off, as the tool is starting to find genuine issues.

Conclusion

Using PBT to test OCaml 5 started as a blue-sky project at Tarides. Despite a range of challenges, the test suite has nevertheless reached a point where a run yields a clear signal free of false alarms and worthy of a confidence increase in a compiler code change. Getting here was a team effort with contributions from Charlène Gros, Samuel Hym, Olivier Nicole, Nicolas Osborne, and Naomi Spargo, along with patience and effort from OCaml compiler engineers working to fix our reported findings.

The PBT approach has successfully pinpointed a range of issues across the Stdlib, the rewritten OCaml 5 runtime system, and the restored backends. We hope that the test suite can continue to do so and thereby help maintain OCaml's reputation as a rock solid and safe platform.

To meet the growing interest in OCaml, we aim to make learning the language as straightforward as possible. For individuals, we’re prioritising tutorials, books, and events centred around OCaml. For teams and organisations, we offer OCaml courses for both basic and advanced training.

The Best Ways to Learn OCaml: For the Independent Learner

OCaml.org

Whether you’re looking to write a ‘hello world’ or delve into the details of the OCaml compiler, the central resource is OCaml.org. The Learn area provides a range of resources, including tutorials, exercises, and a directory of books and academic papers, catering to different skill levels. The page also offers ways to install the latest version of OCaml, access the manual, and explore OCaml’s documentation. The team has taken inspiration from other languages like Python, Rust, and the functional language Haskell to improve the resources available to learn OCaml.

These resources are tailored to the independent learner, using their free time to pursue knowledge, or the teacher needing materials for their students. We support the continuous improvement of tutorials and documentation on OCaml.org, and the accessibility of books, papers, and other learning materials. The pages are also open-source, meaning that everyone is welcome to contribute!

Books

If you prefer learning from books, there are several available to choose from. For beginners, OCaml Programming: Correct + Efficient + Beautiful and OCaml From the Very Beginning are excellent choices. The former is a textbook used at Cornell University, complete with a video playlist. Intermediate learners may find Real World OCaml and More OCaml: Algorithms, Methods, & Diversions beneficial. Real World OCaml is available for free download thanks to our sponsorship of its gold open access release.

Community

OCaml’s robust open-source ecosystem significantly aids newcomers by making source code freely available for study, and offering access to experts and feedback through the GitHub repository and Discuss forum.

There are many more ways to engage with the OCaml community online:

• Social media: head over to the Community page on OCaml.org for an overview of the different places online people meet to discuss, develop, and blog about OCaml. Resources include more or less formal forums, educational text- or video-based content, entertainment, social networking, and live chats.
• Advent of Code: An advent calendar of programming puzzles, suitable for a variety of skill levels and compatible with all languages, popular with programmers as a way to challenge themselves and learn new languages. This year, Sabine has organised a leaderboard for people solving the advent of code in OCaml.
• The OCaml Planet: An RSS feed aggregator hosted on OCaml.org which shares articles and blog posts on everything OCaml. Follow it to stay up-to-date on the latest OCaml news, technical deep dives, open-source project updates, and developer insights.

Events

We also host and sponsor community events like hacking days and conferences to further support users who want to learn more about OCaml. Upcoming events include:

• The OCaml Workshop: Continuing in a well-established tradition, the OCaml Workshop will be held at the 2025 ICFP Conference. With a more academic focus, talks start out as papers which are then presented as part of the OCaml track.
• The FUN OCaml Conference: Returning for a second time in 2025, the new OCaml conference is organised by developers for developers and covers topics big and small.
• Reason OCaml India Meetups: A community of Reason and OCaml enthusiasts that organise regular in person and online meet-ups.

There are of course many more events organised outside of those we host and sponsor, and some notable examples are Functional Conf and OUPS. We recommend that you check out the Events page to get an overview of all the great OCaml events you could join!

The Best Ways to Learn OCaml: Tailored Training in OCaml

In large organisational teams, relying on individual learning or the dissemination of knowledge from a few experts can be time-consuming and inconsistent. Tailored courses enable simultaneous training for the entire group, fostering collaboration in tackling new challenges. To make learning OCaml easier for groups, we’ve created courses to enhance team competence in the language. Here’s what our OCaml courses offer.

Our Foundational Course Starting with OCaml: An Introduction

This course teaches core OCaml programming concepts through theory and practice, and has been designed to facilitate a smooth transition for engineers from other languages, including imperative programming. It covers OCaml’s history, design choices, and key functional programming concepts such as functions, recursion, and higher-order functions alongside critical concepts like tuples, the OCaml type system, type inference, pattern matching, and polymorphism.

The bulk of the course comprises several modules that introduce the practical building blocks of programming with OCaml. This includes methods: imperative and modular programming, data structures, error handling, interfacing with C, and command-line parsing. It also offers guidance on tooling: opam, Dune, VSCode, debugging and profiling, concurrent programming with lwt, and testing using expect tests and QuickCheck.

Finally, the course would not be complete without hands-on experience writing OCaml code. The modules on setting up the OCaml environment, building, documenting, and releasing an OCaml project culminate in an entire day spent on building an OCaml application from scratch. Participants will leave with a solid understanding of OCaml, the necessary tools for development, and practical application-building experience. Following this foundational course, your entire team will be familiar with the OCaml language and workflow.

Our Higher-Level Course Mastering OCaml: Advanced Techniques

The advanced course is designed for experienced users and teams looking to enhance their skills. It consists of three sections: advanced OCaml techniques, advanced methods for common OCaml tools, and advanced tooling.

Advanced OCaml techniques make up the bulk of the course, including generalised algebraic data types (GADTs), multicore programming with OCaml 5, and transitioning from OCaml 4 to OCaml 5. Advanced methods for common OCaml tools cover using the Dune build system, and PPX preprocessors.

Topics in advanced tooling include: the MirageOS library operating system for secure, high-performance, network applications; web development with JS_of_ocaml, Wasm_of_ocaml, and Dream; concurrent programming with [Eio], and using property-based testing to evaluate programs. This course aims to equip teams already using OCaml with new skills and tools. An advanced course with deep topics, the course is tailored to focus on relevant areas for your team, taking them to the next level.

Finally, one of the biggest benefits we offer with our courses is customisation. We understand that teams have unique challenges and goals, and offer you the option to create a course that fits your needs. Not only can you pick a selection of topics from the advanced OCaml modules, but you can completely tailor the training with a customised itinerary. This is negotiated on a per-instance basis, and you can schedule a free consultation to discuss what that could look like for your teams.

Get in Touch

Need help learning OCaml? Reach out to the community on Mastodon, X, or LinkedIn to get helpful pointers or ask questions on Discuss. If you’re interested in our courses for your teams, you can get a free consultation to discuss whether the curriculum is right for you and sign up to our mailing list to receive the latest news.

We look forward to hearing from you, please don’t hesitate to get in touch!

This post will give you an overview of the library, its main features, and some use cases. The team encourages you to try the data structures and share your feedback in the repo and on Discuss. After you're finished with this post, I also recommend you read the paper on Saturn from the OCaml workshop at ICFP 2024 if you want more details. Let’s dig in!

What is Saturn?

The Saturn repository is made up of one package, saturn, containing all lock-free data structures. You can use Saturn with OCaml 5.2 or later, and it can be installed from opam with the command opam install saturn.

Saturn covers many use cases, from simple stacks and queues to more complex structures, including skip lists, hash tables, work-stealing deques, and more. All of them have been adapted to be compatible with and take advantage of the OCaml 5 memory model. For example, the team behind Saturn had to rework the Michael-Scott queue to avoid memory leaks.

To improve performance, they introduced several micro-optimisations, including preventing false sharing, adding fenceless atomic reads where possible (improving performance on ARM processors), and avoiding extra indirection in arrays and atomics to reduce memory consumption. While optimising the library, their work highlighted some missing features in OCaml 5 and led to upstreamed improvements to the language, such as padded atomics and fixing a CSE bug.

Why Saturn?

Sharing data between multiple threads or cores is a well-known problem in computer science. The most obvious solution is to use a sequential data structure protected by a lock, but this approach can introduce performance overhead due to contention between locks. Liveness issues like deadlock, starvation, and priority inversion are also associated with locks in parallel programming.

The opposite approach is to use a lock-free implementation, relying on fine-grained synchronisation instead of locks. This approach benefits from higher performance and guarantees system-wide progress. However, it does come with its own set of bugs, including the ABA problem (which is largely mitigated in garbage-collected languages), data races, and unexpected behaviours as a result of non-linearisability.

In this quagmire of pitfalls, Saturn stands out as a source of reliable, tested data structures that the user can adopt for their own projects. Developers can pick from a variety of structures knowing that they are suitable and safe, saving them time and making OCaml 5 easier to use.

Benchmarks

The library is still relatively new, so more benchmarks are still to come, but there are some numbers to give you an idea of performance. You can find tables showing the throughput of different queues and stacks on single and multiple domains in the paper from the OCaml Workshop at ICFP 2024 about Saturn. The tests reveal that Saturn implementations either outperform or, in the case of simpler implementations, match the performance of non-Saturn structures. However, the benefit of using Saturn, even in cases where another simple implementation exists, is that the Saturn data structures have built-in optimisations and synchronisation safeguards, and it’s up to each individual developer to decide what that’s worth to them.

The library also has a benchmarking command make bench that can be run from the root of the repository to run a standard set of benchmarks. It outputs in JSON and is intended to be consumed by current-bench.

Tests

Testing Saturn’s structures was a high priority for the team and crucial to ensure the safety, linearisability, and lock-freedom where expected. Saturn was tested using two primary tools: DSCheck and STM. STM is used for both unit testing and linearisability. It can automatically generate random full programs using the API provided, and for Saturn, this is a data structure. It then executes the programs in parallel with two domains and checks all the results against the post-conditions of each function, providing unit testing. STM performs a sequence of random commands in parallel, records the results, and checks whether the observed results can be linearised and reconciled with some sequential execution.

DSCheck is a model checker designed to compute all the possible interleavings of instructions between multiple domains and verify that each one returns the expected result. This method is useful for detecting bugs that only occur in rare circumstances which would otherwise be hard to catch. DSCheck can also be used to verify that a program is lock-free, as it will fail to terminate if any form of blocking is present in the program. The DSCheck implementation has been optimised to make the tests efficient even on Saturn’s more complex data structures.

Formal Verification

The verification work for Saturn is done at Inria, forming part of Clément Allain's PhD work there. Due to the notoriously finicky behaviours of lock-free algorithms, they have formally verified part of Saturn’s data structures and aim to keep going until they’ve covered the entire library. The main criterion for correctness in concurrent data structures is linearisability, which requires each operation in a data structure to appear to take effect instantaneously at some point during its execution (called the linearisation point) so that all linearisation points from all operations form a coherent, sequential history. The team at Inria are formally verifying that this linearisability is correctly present in Saturn's data structures.

To verify linearisability, they use the mechanised concurrent separation logic IRIS, which has been used in the past to verify realistic data structures. All proofs are formalised in Coq and available on GitHub. They manually translated the original code from Saturn to a deeply embedded language in Coq, hoping to potentially automate the process in the future. Work continues to verify data structures for Saturn, ensuring safe and predictable behaviours.

Stay in Touch!

To delve deeper into everything Saturn, I recommend you watch the ICFP 2024 presentation from the OCaml Workshop. You can try Saturn for your projects via the repo, and don’t forget to share your feedback with the team!

Connect with us online on Bluesky, Mastodon, Threads, and LinkedIn to stay updated on our latest projects.

In this post, we delve into more detail concerning the design choices behind Notafs and provide some benchmarks and visualisations to give you a better understanding of the file system. Let's dig in!

Big Design Decisions: Copy-on-Write

The Limits of File Systems

In our previous post, we mentioned some of the expectations of file systems to resist corruption and data loss. Ensuring these conditions are met can be challenging since hardware exhibits many failure modes. For example, it is impossible to check if a disk has physically persisted a write operation. In order to improve performance, disks use RAM buffering to quickly store the write requests before committing them persistently (which is a much slower process). Short of unplugging the disk and waiting for its internal battery and RAM to clear before rebooting, we can't know that the writes were executed and must be ready for the worst-case scenario: unwritten, partially written, or corrupted data regions on the disk! Writes could also be reordered, leading to consistency issues.

Furthermore, the way that disks are structured presents its challenges. Disks are organised into blocks of fixed sizes, like 512 bytes, 1 kb, or 4kb, depending on the model. A block is the smallest unit we can read or write, so even if we only want to update a few bytes on the disk, we still need to re-write the full surrounding block data. Different disk models can have additional constraints, such as:

• Hard drives that favour contiguous block reads and writes to ensure high performance since spinning their head to random locations is expensive.
• SSD exhibits faster wear if one block is updated more often than the others, leading to a shorter device life than what's expected under uniform usage.

All About Copy-on-Write

Considering these factors, Notafs was designed as a copy-on-write file system. This strong design principle means that when we update the file system, we never override a block in place but rather write the new data block in a new, unused location on the disk. Preserving the old block means that Notafs maintains access to a backup until the block is reused. If a write operation fails, for example, in case of a power outage in the middle of multiple block updates, Notafs can recover information from the old block.

This strategy does not impact performance since writing the new data in a new location or in place is virtually indistinguishable from the disk's perspective. From an implementation standpoint, copy-on-write is very similar to how purely functional data structures work to provide fast updates. Only modified data needs to be copied, so fast updates are still possible, as sharing unmodified content between revisions is free.

To implement the copy-on-write design, we needed to keep track of the unused free blocks where writes can be performed. This is akin to the challenge of implementing a memory allocator, but it is made slightly simpler because we only need to allocate fixed-size blocks. Our block allocator is backed by a queue containing the list of free block identifiers. After formatting a disk, the queue is (virtually) initialised to contain all block locations except a couple of reserved blocks. When a new block needs to be written to the disk, we pick a free block location from the queue where the write should take place. Filesystem operations are carefully implemented to free old blocks which aren't reachable anymore by pushing their block identifier location back onto the queue. Once the disk is full, these free blocks will eventually be reused to store new data. At the same time, old data is kept intact and can be used as a backup if needed. To avoid complications in determining if and when a block can be freed, we enforce strict linear ownership of blocks so that we do not need any reference counting or advanced garbage collection.

The queue of free blocks is part of the file system state and must be recoverable when we boot it. This is why the queue state itself is backed by a copy-on-write data structure, which is stored on disk! To ensure good performance, the block locations that have been freed during a single file system operation are sorted before being pushed to the queue. This process favours the allocation of contiguous block locations, leading to faster writes on hard drives. Consecutive block locations are also compressed in the queue by representing them as intervals instead of listing all the intermediate block locations.

When the disk gets full, the previously freed blocks will eventually be reused, storing new data. When that happens, the queue guarantees that we use the blocks that have been free for the longest time so that only the oldest backups become unrecoverable. However, since reusing blocks can lead to data consistency issues if some writes fail to be committed on disk, we needed a way to detect whether a block contains the expected latest data or the old partially-written/corrupted data. To this end, we use a checksum of the stored data to validate each block. We could have stored the checksum in the block next to the data it validates, but it would only have allowed for the detection of corrupted data and not for detecting whether the block's data is old since the old checksum remains valid. So, instead, we store the checksum in the parent block next to the block location it validates. When a parent block needs to read one of its child block's data, it uses its checksum to verify that its child data is as expected.

Final Considerations

There was one last issue for our team to tackle regarding the design: if everything is copy-on-write and the new filesystem root is written to a new block every time, how do we find the location of the latest root when the unikernel boots? We wanted to avoid scanning the whole disk in search of the latest root. Hence, when formatting a new disk, we reserved a fixed number of blocks at known locations. After each file system update, one of these blocks is updated in place to point to the latest root allocation in a round-robin fashion, with a generation counter to help our application determine which one was written last.

When the filesystem starts, it identifies the latest root from the reserved blocks and traverses all of its reachable data to check each block checksum and validate that the whole filesystem is in a consistent state. If an issue is detected, implying the disk wasn't shut down properly, Notafs will attempt to use the previous filesystem root found in the previous reserved block. This procedure effectively rolls back the failed write operation until a valid filesystem state is found (or the disk is deemed unrecoverable!). To avoid SSD wear from updating the reserved root blocks too often compared to other blocks, we have added an extra, dynamically allocated level of indirection to the previous explanations, which amortises the number of updates to the reserved blocks.

Finally, we implemented several optimisations to the above scheme to best use runtime resources. For example, we save disk space by representing block locations with as few bytes as possible, depending on the disk size. A 'Least-Recently-Used (LRU) cache keeps track of the latest reads and bufferises the incoming writes. This LRU enables Notafs to run with very low memory overhead, even when a file system operation would have required more RAM if another system had performed it.

We are very happy with this design because it combines simplicity with high data consistency guarantees. We used the setup to implement a copy-on-write rope data structure to represent files of any size with fast updates and read operations. We added a rudimentary hierarchical structure of folders and files on top to enable multiple file usage. This last part has not been optimised and is only suitable for applications with a low number of files. It will get slower the more files you have, but there are no hard limits.

Benchmarks and Tests

Since filesystem correctness is critical, we tested all steps of the project to validate our solution. Of special interest is the final verification that our team realised with the Gospel specification language using Ortac to translate the specification into a QCheck test suite from the formal spec. By comparing the behaviour of our Notafs file system with an obviously correct in-memory reference model, we could stress-test our system by simulating a huge number of tests to detect discrepancies. Our experience with Gospel made a strong case for it being the future of documentation, with high-level specifications that are both mechanically verifiable and readable by end-users.

Ultimately, correctness alone was not enough to justify using Notafs in production if it did not meet our performance goals. We benchmarked Notafs in comparison to existing MirageOS file systems for the features that were critical for SpaceOS and Irmin support. Those features were the ability to write large files, to read back those files, and to read only a sub-range in those files (a feature required by irmin-pack).

While the TAR file format will always be the fastest thanks to its simplicity, it comes with the limitation that files can't be removed or updated (hence eventually running out of disk space!). In general, the benchmarks confirm that Notafs meets its goal of handling large files efficiently. In particular, its careful use of memory with LRU enables Notafs to run the benchmarks on larger files than the alternative – for unikernels with restricted max memory usage. Note that benchmarks are always subject to interpretation; we did not benchmark the scenarios where we expect Notafs to perform the worst, e.g. with a large number of small files!

Our next image is actually a video showing Irmin running on top of a small Notafs disk. It combines everything we have explained so far: (epilepsy warning: the video displays flashing lights)

• The disk is divided into blocks of 1024-sized bytes, conveniently represented by 32x 32px squares. The blocks in colour contain live data used by the Irmin store, while the grey crossed-out ones are the free blocks. The colourful blocks create distinctive patterns that indicate the different types of data that Irmin uses to store the database.
• As more Irmin operations occur during the test's execution, you can see the Notafs block allocator in action, circling around the disk to write new data to the oldest freed block.
• At the bottom of the video, two histograms show the distribution of the reads and writes on each block to check that fair usage of each block is achieved.
• Finally, we can see the Irmin garbage collector clearing its old history on a frequent basis to avoid running out of disk space.

This visualisation was made possible because MirageOS libraries can also be used on standard operating systems without requiring special compilation by a unikernel. In other words, Notafs also enables Irmin to be used as a single-file database (like sqlite, which might be of interest for some applications' workflows. We want to do more work on this front to lift the limitation that the database size must currently be fixed in advance when formatting the disk.

Conclusion

Developing a new filesystem has been a very exciting project, and we hope that you'll enjoy the new options when developing unikernels with MirageOS:

• Notafs: for applications which require a limited number of very large files (e.g. SpaceOS satellite pictures).
• Irmin on Notafs: for a general-purpose file system, including optimisations to support a large number of arbitrarily sized files and many advanced features (with the caveat that this version of Irmin requires OCaml 5, which is still experimental on MirageOS).

We invite you to participate and try Notafs yourself! Please consult the open-source Notafs repository for further information, and share your experience (good and bad!) with the team. You can stay in touch with us on Bluesky, Mastodon, Threads, and LinkedIn.

MirageOS allows you to compile OCaml applications into unikernels. By selecting the operating system functionalities required, a unikernel can be constructed for your application using only the necessary components. This process reduces the attack surface and increases the hardware efficiency of your final application. MirageOS unikernels can be deployed to various cloud and mobile platforms.

As a case in point, Tarides is developing SpaceOS to run unikernels on satellites, an industry where both security and performance are critical. While MirageOS has a long history of cloud usage and comes with advanced network capabilities, SpaceOS provides an interesting use case for disk storage. A possible use case for satellites is to take pictures from space and send them to Earth for analysis, and since satellites may not be able to send the pictures right away (due to their location, for example), storing high-resolution images on a disk is a must.

So, naturally, the question we posed ourselves was what MirageOS file system could we use for SpaceOS? And that started us down the journey to the new file system Notafs, that even lets you use Irmin on top!

Why Create a New Filesystem?

At the start, there were a couple of existing filesystems available for MirageOS that we needed to evaluate:

• The Chamelon filesystem is designed to efficiently support a large number of very small files on a disk.
• The Docteur filesystem which provides read-only file compression.
• The FAT filesystem, which provides support for the (old) standard FAT16 filesystem with restrictions on file sizes.
• The TAR file format, which has the limitation that users can only add new files: deletion or modification of existing files is not supported.

All of the above libraries are well-tested and highly recommended if they fit your application needs! But, as we will illustrate in part 2 with our benchmarks, none of these systems satisfied the SpaceOS requirement to store large files.

Furthermore, using a conventional file system like the ones available on traditional operating systems like Linux and Windows was also not an option since they are closely tied to large operating system functionalities. This is because to ensure higher security, MirageOS libraries are built using the OCaml programming language, which provides strong memory-safety guarantees. While this is a fantastic design choice for the future of computing, it is harder to support a conventional filesystem (programmed in C/C++) natively on MirageOS. A unikernel based on that kind of technology would lose the appeal of a small software stack with high-security guarantees. In comparison, the Notafs solution fits into four thousand lines, which is a much more ‘human-sized’ project to review.

The Challenges With Filesystems

Implementing a filesystem is a complex task requiring compromises, which explains the lack of options for storing large files with the SpaceOS project. For example, even though file systems may appear simple, with an interface that everyone is familiar with, their critical main function is to protect applications from (recoverable) hardware defects. Unless the disk dies, the user should never lose or see corrupted data, even if a power outage was to interrupt the filesystem in the middle of an operation. In other words, a filesystem update should have transactional semantics: either the operation succeeds, or it does not, but applications should never observe an in-between broken state. Without this property, software built on top of the file system would be vulnerable to experiencing faults.

Tarides is well aware of the challenges involved with implementing a filesystem – we maintain the Irmin database, which provides a hierarchical key-value store with high data consistency guarantees, git-inspired history for rollbacks, and distributed replication over the network. While the Irmin API resembles a filesystem, it is a full-blown database and provides many more functionalities. So rather than starting a new filesystem implementation from scratch, we asked ourselves whether we could reuse the Irmin database as a filesystem for MirageOS.

Irmin as a Filesystem?

Using Irmin for this purpose is not a new idea, and Irmin already provides multiple backends that allow users to run its databases on various platforms with different constraints. The irmin-pack backend was of particular interest for MirageOS support. We have spent years optimising its performance since this backend is notably used to store the Tezos blockchain, and it enables the freeing of disk space by truncating the database history to get rid of unnecessary old backups.

Finally, the low-level implementation of irmin-pack is especially suited for a port to MirageOS, as it only requires support for a few large (append-only) files from the operating system.

This last technical requirement was especially relevant to the SpaceOS use case, where the satellite needs to be able to store high-resolution pictures on disk. If our custom-made file system supported large files, then we would be able to support SpaceOS and enable the general-purpose use of Irmin for MirageOS unikernels. This realisation still left us with the task of developing the foundations of a file system, but even just the bare functionalities would satisfy our use cases.

Notafs is Born

We called this new file system ‘Notafs’, which, as the name suggests, is not a general-purpose file system due to its focus on handling a few large files. It is designed to handle a small number of large files for Mirage block devices. It can, however, be used to run the irmin-pack backend, which gives users all the benefits of an Irmin filesystem for MirageOS. Together, running irmin-pack on Notafs lifts the limitations of the latter, and supports many file names, is optimised for small and large files, and includes a git-like history with branching and merging just to name a few features!

Navigating design restrictions is an excellent way to focus the efforts of a project. In our case, that meant directing them towards the correctness and performance of the few selected operations of Notafs. We didn’t need to implement advanced filesystem operations since missing functionalities could be provided by irmin-pack, including optimised management of many small files. As long as our underlying file system could provide fast operations on large files, the rest was taken care of!

From a unikernel developer’s perspective, Notafs provides an implementation of the Mirage_kv interfaces. This is the standard API for filesystem usage on MirageOS, so any existing unikernel can use it without needing to change its application code. We also provide a command-line interface to simplify the formatting of disks and to allow for external inspection of file system contents of the unikernel. Check out this example of a minimal setup of Notafs in MirageOS.

While Notafs has restricted functionalities, it provides a nice developer experience and a useful alternative in the file system design space for MirageOS. We are especially proud of the safety guarantees that Notafs provides for your data stored on disk.

Using Irmin on Notafs

To enable users to run Irmin on MirageOS, we provide an implementation of the syscalls required by the irmin-pack backend on top of Notafs. This was straightforward as Notafs provides the functionalities required to run irmin-pack. Thanks to OCaml functors, the irmin-pack codebase was already structured for alternative syscall implementations.

One OCaml subtlety we encountered was handling asynchronous I/O: MirageOS and Notafs use the Lwt monad, while the Irmin syscalls expected a direct-style I/O implementation. This would have been an issue in the past, but we could bridge that gap using the effect handlers that came with OCaml 5. Note that OCaml 5 support is still experimental on MirageOS at the time of writing!

From a user perspective, using Irmin addresses the design limitations of Notafs (while keeping its desirable consistency properties) by adding efficient support for managing a large number of small files. Irmin is a much more general-purpose file system; on top of that, a user can build a unikernel application with many additional features. We’re hopeful that applications already using Irmin will consider this new bridge for targeting MirageOS unikernels.

Until Next Time!

Thank you for checking out the first part of this two-part series where we’ve introduced Notafs, the motivation behind the file system, the challenges, and its use cases. Look out for part two where we delve into the details behind Notafs’ design alongside benchmarks and visualisations of the file system in action.

Connect with us online on Bluesky, Mastodon, Threads, and LinkedIn to stay updated on our latest projects. We also invite you to consult the open-source Notafs repository for more information or to test out the file system yourself!

The Language Server Protocol (LSP) defines a standardised protocol that facilitates communication between an editor (client) and a language server. It is developed by Microsoft and was designed to simplify the process of adding language support to different code editors by abstracting away the most common implementation details of the programming language.

OCaml-LSP, which relies on Merlin's amazing engine, is an implementation of the LSP protocol for the OCaml programming language.

2. Code Navigation in LSP

When it comes to navigating through code, the LSP protocol provides four ways to do this:

• Goto Declaration Request: Resolves the declaration location of a symbol.
• Goto Definition Request: Resolves the definition location of a symbol.
• Goto Type Definition Request: Resolves the type definition location of a symbol.
• Goto Implementation Request: Resolves the implementation location of a symbol. Not supported in ocaml-lsp.

While these goto requests provide general navigation for most programming languages, it falls short when dealing with complex syntax such as OCaml's modules, functors, etc., and being able to navigate to specific sections.

3. Code Navigation in Merlin

Merlin offers a range of commands that allow precise code navigation, including the ability to jump to specific constructs like functions, module definitions, and match cases. As earlier seen, generalised navigation in LSP is insufficient for precise movements. To solve this, Merlin introduces a specialised Jump command that allows users to jump to specific targets within their OCaml code.

The key targets include:

• fun: Jumps to a function definition.
• let: Jumps to a let binding.
• module: Jumps to a module.
• module-type: Jumps to a module type definition.
• match: Jumps to a match construct.
  • match-next-case: Jumps to the next case in a match statement.
  • match-prev-case: Jumps to the previous case in a match statement.

4. Custom Requests in LSP

When standard LSP requests are insufficient, we have the possibility of writing custom requests to implement the functionality we want. The downside to this approach is that we lose native support in the various editors or clients and will also have to write the resulting client implementations for each custom request. Implementing precise navigation in OCaml-LSP using Merlin's Jump is a typical use for a custom request.

For this implementation, our focus was on being able to use the requests already available on LSP and use them in non-typical, innovative ways. With this in mind, our solution works by using a CodeAction in combination with a ShowDocument request to move the cursor to our desired position.

5. Implementing Merlin's Jump in OCaml-LSP

Code action requests are used to execute commands for a given text document and range. These commands are typically used to change the state of a document, such as code fixes and beautifying or refactoring code. In general, we use code actions to perform quick edits in code. It's a bit unusual to think of code actions as a navigation tool, but with some smart workarounds, we can indeed use code actions to move through code. Here's how the feature works:

• a) Document Detection: When a source file is opened in the editor, the OCaml-LSP server checks if it is an OCaml file supported by Merlin. If the document is incompatible, the "Jump to Target" functionality is not provided.

• b) Client Capability Check: The LSP allows servers to query what features a client (the editor) supports. For "Jump to Target" to work, the server checks if the client supports the ShowDocument capability, which is necessary to move the cursor to the correct location.

• c) Generating CodeActions: The client begins by sending a request for all valid code actions:

[Trace - 06:21:32] Sending request 'textDocument/codeAction - (71)'.
Params: {
    "textDocument": {
        "uri": "file:///.../test.ml"
    },
    "range": {
        "start": {
            "line": 5,
            "character": 12
        },
        "end": {
            "line": 5,
            "character": 12
        }
    },
    "context": {
        "diagnostics": [],
        "triggerKind": 2
    }
}

The server receives this request, and for each target (fun, let, match, module, etc.), it asynchronously queries Merlin's Jump command. It assembles the possible jumps into a list of code actions and returns this list back to the client.

[Trace - 06:21:32] Received response 'textDocument/codeAction - (71)' in 7ms.
Result: [
    ...,
    {
        "command": {
            "arguments": [
                "file:///.../test.ml",
                {
                    "end": {
                        "character": 2,
                        "line": 9
                    },
                    "start": {
                        "character": 2,
                        "line": 9
                    }
                }
            ],
            "command": "ocamllsp/merlin-jump-to-target",
            "title": "Let jump"
        },
        "kind": "merlin-jump-let",
        "title": "Let jump"
    }
]

Given that code actions are a standard LSP functionality, we benefit from an already existing pool of client implementations, so the clients display the codeactions returned by the server without any coding or customisation. Each CodeAction includes a:

• Title (e.g., "Jump to function").

• Command (ocamllsp/merlin-jump-to-target): A command that is executed when the code action is selected.

• Kind: This parameter distinguishes various code actions and can be used to group similar code actions or differentiate them especially for use with keybindings.

• d) Executing CodeAction Commands: When a user clicks or selects a specific code action, the command associated with that code action is executed. The client sends an executeCommand request to the server.

[Trace - 06:37:10] Sending request 'workspace/executeCommand - (73)'.
Params: {
    "command": "ocamllsp/merlin-jump-to-target",
    "arguments": [
        "file:///.../test.ml",
        {
            "end": {
                "character": 2,
                "line": 9
            },
            "start": {
                "character": 2,
                "line": 9
            }
        }
    ]
}

This executeCommand request triggers the server to ask the client to perform a showDocument request using the document URI and location range received from the code action.

[Trace - 06:37:10] Received request 'window/showDocument - (6)'.
Params: {
    "selection": {
        "end": {
            "character": 2,
            "line": 9
        },
        "start": {
            "character": 2,
            "line": 9
        }
    },
    "takeFocus": true,
    "uri": "file:///.../test.ml"
}

Both executeCommand and showDocument are standard LSP requests, meaning all clients that already have LSP support can automatically perform this jump.

• e) Error Handling: In cases where Merlin fails to find a valid target (e.g., the target does not exist or the position is invalid), the server gracefully handles the error by omitting the specific code action for that particular target, ensuring a smooth user experience.

6. Feature Demonstration

Below is a demonstration showing the various code actions which perform navigation.

7. Conclusion

Set for release in early December 2024, Merlin's Jump functionality in OCaml-LSP brings precision navigation into the LSP world, enhancing OCaml code navigation capabilities across various editors. By using a combination of existing LSP requests, the feature is implemented in a way that is client-agnostic and fully compatible with LSP, ensuring a smooth and efficient user experience for developers working with OCaml.

For developers working on large or complex OCaml projects, this precise navigation capability will significantly streamline workflows, enabling quick and direct navigation to relevant parts of their code.

While implementing Merlin's Jump as a code action offers a broad client-agnostic solution, there are several drawbacks to this approach. First, it can clutter the code action lists, making them noisy for users who are used to a more streamlined selection. Since code actions are typically used for refactoring or fixing issues, adding navigation-related actions here is not an expected usage of code actions. Additionally, binding these navigation actions to shortcuts becomes harder, as code actions do not naturally lend themselves to precise key bindings for navigation.

To overcome these limitations, one potential solution is to use a custom LSP request, which would allow navigation functionality without overloading the code action list. We have a an open PR on ocaml-lsp repository which introduces this custom request. Alternatively, a client-side tool like tree-sitter could be used to parse the code and generate the necessary jump targets directly in the client, offering a more flexible and efficient solution tailored to the user's editor setup.

8. Resources

• LSP Protocol - ShowDocument Request
• LSP Protocol - ExecuteCommand Request
• LSP Protocol - CodeActions Request
• Issue Tacker - Merlin
• Issue Tacker - OCaml-LSP

As a key sponsor - alongside Ahrefs and Jane Street - Tarides is striving to make OCaml more accessible. I have had the pleasure of interviewing one of the head organisers Sabine Schmaltz about her experience at the conference, the aim of the event, and the future of FUN OCaml. Let's dive in!

Why FUN OCaml

FUN OCaml began as a small idea that grew thanks to great teamwork; as Sabine recalled, "David Sancho from Ahrefs was thinking of organising an online conference and had already done a lot of work to plan the talks, etc. I reached out to explain that I was planning an in-person conference and asked whether we could join forces – and he immediately said yes!" It's great to see collaboration across companies benefiting the community, and several additional volunteers would join the organising committee as the conference drew nearer.

I was curious to know what motivated Sabine to organise the event, and she explained that:

"I felt that OCaml was missing a bigger event that was targeted uniquely at developers. FUN OCaml gives them a space to teach each other things and build connections, bringing maintainers and contributors together. FUN OCaml is organised by developers for developers, covering things they are interested in on a peer-to-peer basis".

She was inspired by ZuriHac, a yearly Haskell event in Zurich that she has attended before. "It has a great spirit; people go there to hack and hang out by the lake. It's a very social event with many different rooms and discussions". Inspired by ZuriHac, Sabine adopted an alternative approach to scheduling for FUN OCaml in comparison to traditional conferences.

"We didn't stop at 6 pm and send everyone away! We stayed open until midnight on the first day and at 10 pm on the second. I wanted to provide a space in the evenings for activities, whether OCaml-related or not. Letting people put faces to names and form relationships was important because it's crucial for collaboration". The evenings were filled with board games, hacking, hanging out, and karaoke – creating a space for socialisation and mingling. Sabine pointed out that when conferences close after their programs end, people generally stick to groups of people they already know to hang out with in the evening, which prevents them from making new connections. Consequently, creating a social space for everyone was a high priority for the organisers and a core motivation behind the conference.

Talks, Workshops, and More!

So, what was a day at FUN OCaml like? Both days ran a track for talks and a track for workshops. The talks were live streamed and are [available online for you to watch]. Recording them meant that participants who went to a workshop could catch up on the ones they missed after the fact. The talks covered MirageOS, GADTs, odoc, type engineering, OCANNL, and learning OCaml with Tiny Code Xmas, just to name a few! It was a great mix of more and less technical topics. Sabine recalled the software engineer and popular live streamer Dillon Mulroy's presentation: "Dillon gave a great talk on his journey to OCaml, like a personal account of how he came to OCaml and the OCaml community. It was great to see a Typescript developer and public figure in the developer community talk about OCaml".

Dillon commented on his time at FUN OCaml:

“FUN OCaml was my favorite conference I've ever attended. I finally got to meet so many friends and peers that I've met online over the past year or two and got so much value out of collaborating with them in person. I can't wait for next year 🐫”

The workshop track was a valuable addition that gave participants an alternative way of engaging with OCaml: "People really enjoy the hands-on approach and the rare opportunity to hack on projects together with their maintainers". The workshops covered a range of topics such as a beginner's introduction to OCaml, building reliable actor systems, concurrency and parallelism, MirageOS, creating 2D games in OCaml, and web and mobile app development. "I heard from the host of the game engine workshop, Émile, that people built many cool things, including a minesweeper game!"

FUN OCaml 2025 and Running Your own OCaml Meet-up or Conference

To wrap up, I was curious about what Sabine had planned for the future, including whether there were any plans for another FUN OCaml. I am pleased to report that FUN OCaml is on track for 2025!

"This year, we had to schedule FUN OCaml for September to make it work with ICFP and REACT Alicante, but that meant it fell outside of the semester break for many students. Next time, we would like to schedule FUN OCaml between semesters so as many students as possible can come. They would benefit from hacking on open-source projects and getting feedback from maintainers. Software in the wild!"

We want to see more OCaml events around the world! You can organise your own conference or meet-up to bring more OCaml to your neck of the woods. Sabine is happy to help any member of the OCaml community looking to create events, and you can contact Sabine on X, on Bluesky, or by email.

Until Next Time!

Connect with us online on Bluesky, Mastodon, Threads, and LinkedIn. We look forward to hearing from you and hope to see you around the OCaml universe!

OCaml’s success is the combined result of the innovative efforts, passion, and dedication of the people who contribute to its development. The open-source community behind OCaml works collaboratively and transparently to make OCaml faster, safer, and easier. This community is made up of individual developers, research groups, and companies all working together. Tarides is one of the companies that contribute extensively to OCaml, and several of our team members are core OCaml developers.

You can support our open-source work by becoming a sponsor on our GitHub page. Your contribution will have a direct impact on the OCaml ecosystem by helping us maintain, develop, and improve its libraries, features, and tools. Let’s show you what we’re working on and what your support will help us achieve for OCaml!

What Does Tarides do?

At Tarides, we’re passionate about making OCaml even more powerful and accessible for developers. Our work focuses on three key areas: Enhancements, maintenance, and accessibility. Your contribution will support our work in all these areas.

1. Enhancing the Language: We bring new features to OCaml by resolving developer pain points and testing performance on different platforms. We want OCaml to be a competitive programming language with developer experience a high priority.
2. Maintaining Core Tools and Libraries: We ensure that OCaml developers have a reliable foundation for their projects by keeping the tools and libraries they depend on up to date.
3. Community Support and Outreach: We prioritise clear documentation and tutorials to improve accessibility and regularly organise events that allow community members to meet and exchange ideas.

What Projects Does Tarides Work on?

We work on a broad range of projects targeting different parts of the OCaml ecosystem: Compiler and language tools, development tools, community and infrastructure, and advanced projects.

Compiler and Language Tools

• OCaml Compiler: Long-term maintenance of the compiler alongside several collaborators, as well as feature development and enhancements.
• OCaml 5: We’re driving the OCaml 5 release with Multicore support and effect handlers.
• Js_of_ocaml and wasm_of_ocaml: Run OCaml code in your browser!

Development Tools

• OCaml Platform: Core OCaml tools, ensuring availability and compatibility with new compiler releases.
• VSCode extension: Support and development of the OCaml Platform VSCode editor extension.
• opam: The OCaml package manager, its tools, and plugins.
• Dune and Dune Developer Preview: The OCaml build system.
• Merlin and OCaml-LSP: A modern IDE for OCaml.
• odoc: A documentation generator for OCaml.
• OCamlFormat: Formatting for OCaml code.

Community and Infrastructure

• OCaml.org: OCaml’s home on the web, the central knowledge base where the community can connect, access resources, and get the latest OCaml news.
• OCaml Infrastructure: Maintenance of various parts of the OCaml.org and opam infrastructures.
• OCaml Package Ecosystem: Maintenance of the growth and quality of the OCaml package ecosystem, including opam-repository and related opam-repo-ci, OCaml-CI as well as opam-health-check for Linux, FreeBSD and Windows as public services.

Advanced Projects

• Eio: A modern, effect-based I/O library for OCaml designed to provide a high-level, structured concurrency model.
• MirageOS: An operating system that constructs unikernels for secure, high-performance applications across various cloud computing and mobile platforms.
• Irmin: Distributed data stores based on distributed version-control systems.

Contribute to the Future of OCaml Today!

By sponsoring us, you’ll support the maintenance of these essential OCaml tools and libraries and contribute to the growth of a diverse, dynamic community. Your contribution will have a direct impact on our projects and the OCaml ecosystem as a whole.

Please reach out to us on Bluesky, Mastodon, Threads, and LinkedIn. We look forward to hearing from you!

To that end, we are thrilled to announce the ARGOS project, which stands for Analyse et Représentation des Graphes des Opérations Suspectes or the analysis and graphical representation of suspicious operations. Together with Functori, the LMF lab at Paris Saclay, and the Lip6 lab at Sorbonne University, Tarides is creating a platform for blockchain analysis. ARGOS will facilitate the monitoring and tracing of individual transactions on the blockchain, helping both nations enforce the law and organisations comply with regulation.

The Current Problem

The financial crimes perpetrated using crypto and the blockchain include money laundering, extortion, fraud, and funnelling funding for terrorist activities. Cybercriminals take advantage of the same traits that make crypto attractive to legitimate users: its decentralisation, dynamism, and versatility. Further exacerbating the problem, these same traits make it difficult for nations to legislate and enforce rules like they do for traditional financial institutions like banks.

This is because, rather than employing the traditional methods of identification, authentication, and document verification by a central authority, the blockchain uses an 'authorisation by default' approach to validate the exchange itself. Even though existing safeguards exist, which make it possible for police and the judicial system to determine the individuals associated with a transaction, they are not enough to effectively tackle cybercrime. What is needed is a way to identify, analyse, and track suspicious transactions and their sources at scale.

The ARGOS Solution

Several proposed solutions to the situation exist, but many of them can't address the breadth and complexity of the challenge or/and are not designed for the European financial system. Existing regulations empower nations to pose specific requirements on cryptocurrency providers and traders, and in Europe, that includes PACTE for France. Regulations impose rules on how the blockchain interacts with traditional financial institutions and the obligations that providers have including authenticating clients, freezing accounts, and identifying cryptographic addresses.

ARGOS will be a French solution custom-created to comply with and enforce French and European financial market regulations. It is designed to be lightweight enough not to place an undue burden on the existing ecosystem of blockchain developers, cryptocurrency providers, and traders, efficient enough to handle large codebases and complex systems, and robust enough to catch criminal activity reliably.

What is ARGOS?

The success of ARGOS rests on the key characteristics of the project, which are as follows:

• Data Sovereignty: ARGOS will be designed and developed in France in compliance with the best practices and regulations of the European market. This provides additional security and privacy guarantees for European users, who do not have to share sensitive information with non-European parties.
• Open Source: Being open-source is a fundamental aspect of the ARGOS project, providing the high levels of transparency required to facilitate collaboration across organisations. The project will be published under an open-source licence, ensuring the entire platform (including the source code, tests, etc.) is available to the community for validation and scrutiny. This transparency is important for users to feel confident in the platform and to encourage innovation in its ecosystem.
• Large-Scale Data Management: We will use and adapt large-scale data management techniques to handle the big datasets characteristic of the blockchain sector. Our focus will be on efficiency and performance, and we will undertake R&D work to develop new solutions that suit our use case.
• Suspicious-Pattern Detection: We plan to implement data processing mechanisms like taint analysis and pattern detection to track the movement of digital assets. Machine learning should also greatly improve the quality of the results we obtain.
• In-Depth Analysis of Smart Contracts: Our system will be able to analyse smart contracts and understand the paths of nested interactions to determine whether a series of operations is suspicious.
• Blockchain Bridge Traceability: Our solution will account for transactions across blockchain bridges, ensuring that ARGOS doesn't lose track of them and remains global (not limited to one blockchain).
• Customisable User Interface: ARGOS will provide an intuitive and customisable user interface that can be adapted to suit each user's individual needs. It's important that our solution is accessible, easy to use, and integrates with existing tools. For example, a Digital Asset Service Provider (DASP) (so-called Prestataire de Service sur Actifs Numériques or PSAN in French) should be able to use it in combination with the monitoring and cryptographic tools they already employ with their own users.
• Adapted Analysis Reports: ARGOS will offer several analytics report tools tailored to the specific needs of different users.

What Does the Future Hold?

The project is in its early stages, and we are collaborating closely with our partners to research and develop new approaches and technologies. Look out for more updates on the project coming in the future!

Connect with us online on Bluesky, Mastodon, Threads, and LinkedIn to stay updated on our latest projects.

Tarides are co-sponsors of ICFP, and several of our team members attend each year. This year, we had six talks given by Tarides team members and many of our colleagues chose to go as participants. I have asked several of them to share their experience at this year's ICFP to give you a taste of what the conference is all about.

The Tarides Talks

Before we get started, let's recap the talks created or co-created by Tarides engineers and where you can watch them after the fact.

• First-Class Windows: Building a Roadmap for OCaml on Windows: A talk introducing the project seeking to make the developer experience with OCaml on Windows as good as it is on macOS and Linux. Watch the first-class Windows talk here.
• Opam 2.2 and Beyond: Gives background to the recent opam release from the perspective of its core maintainers. Watch the opam 2.2 talk here.
• Picos – Interoperable Effects-Based Concurrency: Covers the ongoing project to create an interface between effects-based schedulers and concurrency abstractions. Watch the Picos talk here.
• Project-Wide Occurrences for OCaml, a Progress Report: Describes the design behind the first iteration of improved search features available with editor tooling. Watch the project-wide occurrences talk here.
• Saturn: A Library of Verified Concurrent Data Structures for OCaml 5: This talk covers Saturn, a new library of well-tested, benchmarked, partially verified concurrent data structures. Watch the Saturn talk here.
• Wasm_of_ocaml: Presents the work done on the Js_of_ocaml fork which translates OCaml bytecode to Webassembly. Watch the Wasm_of_OCaml talk here

Experience Reports

Curious about ICFP or missing the action already? I've asked several of my fellow Tarides team members who attended ICFP this year to share their thoughts and experiences. Let's dive in!

Why ICFP?

ICFP stands out to functional programming enthusiasts for gathering a large community of like-minded people in one place. As Jan Midtgaard puts it, "it's a wonderful mix of academics and industry people gathering due to a common interest in functional programming". KC Sivaramakrishnan has been attending ICFP since 2009 when he was a PhD student, and "ICFP is now the place where I meet friends and collaborators as well as the future generation of functional programming researchers".

Another reason behind the conference's enduring popularity is, of course, the packed schedule full of high-quality talks. The different tracks offer a variety of opportunities for participants to explore topics that interest them. Ambre Suhamy comments, "I know for sure that I will learn something new and come back from ICFP with more knowledge".

Stand-Out Talks

On the topic of talks, my colleagues attended several across different days and tracks. I asked them to share some of their impressions from the presentations, and while there were far too many to include them all, let's check out some of what they had to share!

• ICFP Paper: Functional Programming in Financial Markets and OCaml Workshop: Recursion schemes in OCaml: Tim McGilchrist really enjoyed these two talks, the first focussing on Standard Chartered Bank's use of typed functional programming (Haskell) across their entire tech stack, and the second on Bloomberg's use of OCaml modelling bilateral financial contracts. "An interesting experience report for recursion schemes at Bloomberg. I'd previously only seen this implemented in Haskell!"
• ICFP Papers: Oxidising OCaml: To KC Sivaramakrishnan, this was a "technical highlight". The talk centred around the potential for optimising heap allocations in OCaml without compromising on safety guarantees. "The modal types work aims to get the best of Rust into OCaml without letting go of what OCaml is good for – and the Rust-like features are opt-in!"
• OCaml Workshop: Opam 2.2: Both Riku Silvola and Jan were impressed by this presentation on the new opam 2.2 release which notably brought native Windows support among other features. "That Raja Boujbel from OCamlPro, Kate Deplaix from Ahrefs (and the OCaml Software Foundation), and David Allsopp from Tarides gave a joint opam 2.2 talk sent a wonderful signal that three OCaml companies have worked together to do good for the community", said Jan. Riku agreed, commenting "David, Kate, and Raja presenting the long-awaited opam 2.2 release with its plethora of new features highlighted a longstanding and fruitful open-source collaboration across companies". They also announced opam's new time-based release cycle of updates every six months, check out the opam 2.3.0 blog post to learn more!
• ML Track: Pattern Matching on Mutable Values: This one stood out in Ambre's mind for its memorable demonstration of an obscure corner case where the pattern-matching compiler would generate incorrect code. "It was insane; here's this five-line snippet of OCaml code; you can run it today, and it segfaults!" The original bug was found in 2016, and work has been ongoing to narrow down and fix the problem. Even though it was a rare edge case, the community rallied to address it, and a bug fix will be included in OCaml 5.3.
• OCaml Workshop: Project-Wide Occurrences: Riku highlighted the work presented by Ulysse Gérard, introducing a new feature of editor tooling that lets users query all occurrences of a selected value anywhere in their project. "This work also highlights the important part the compiler plays in the overall platform vision, as the feature required work to be done not only in ocaml-lsp-server and merlin but also in dune and the compiler", Riku said.

Memorable Moments

My colleagues' week in Milan created several lasting memories. Ambre chaired her first session, the 3rd OCaml Workshop session, which included presentations on Priodomainslib, Saturn, Picos, and Distributed Actors. She also attended FARM, the international workshop on Functional Art, Music, Modelling, and Design. Ambre remembers the workshop as "people using music to make programs, or programming to make music, and understanding music through programming".

Tim recalls his many conversations between events with a variety of people in the hallways. It might not be the first thing that springs to mind, but the conversations that strike up organically at ICFP can be enlightening. For example, "people generally didn't know that you could use GBD/LLDB on OCaml binaries and, once they knew that, they were very excited about using those tools on their OCaml programs".

KC is focussed on the future, highlighting the feedback he received from participants and how it will help Tarides going forward. "I got some really nice feedback from folks, building on OCaml, about the work that Tarides is doing. I also got lots of honest feedback on what's not working. At the end of the day, our user community matters, and we need to solve their pain points so that OCaml becomes an advantage and not a technical risk for their engineering teams." He also wants to keep contributing to the future of ICFP: "I'm on the ICFP steering committee and will be a DEI co-chair for the next ICFP. I want to ensure that ICFP remains a thriving, friendly, and inclusive place for all of our attendees".

Finally, we can't discuss a Milan conference without mentioning the food! All the participants enjoyed sampling the local Italian fare and just look at this delicious pizza!

Join us Next Year!

The good news is that ICFP happens every year, so if you didn't attend this year, you can always set your sights on next year. The conference also moves around, alternating locations to make it easier for participants around the globe to join. We look forward to seeing you at ICFP 2025 in Singapore, so come find us if you're going!

You can reach out to us on Bluesky, Mastodon, Threads, and LinkedIn. Stay in touch!

Until now, Dune’s Package Management has supported the installation of various packages from the opam repository. However, it lacked the ability to install a functioning OCaml compiler — a crucial component for most Dune projects. This limitation meant that early adopters of Dune’s Package Management had to rely on external tools to manage their OCaml compiler installations, which wasn’t ideal. The good news is that this obstacle has been removed, making Dune Package Management far more robust and ready for testing by early adopters.

The challenge we faced in integrating the OCaml compiler installation stemmed from a conflict between the compiler’s build system and the way Dune handles package builds. Dune builds packages in what’s known as a “sandbox” — a temporary directory where a package is initially constructed and installed. Once the build is complete, the package's installed components are moved to their final destination within Dune’s build directory. However, the OCaml compiler assumes that its installation location will be its permanent home. Moving the installed files after installation caused the compiler to malfunction, making this an intractable problem for Dune’s package management.

While work is underway to make the OCaml compiler more flexible in terms of its installation location, we didn’t want to delay Dune’s package management features until this work was completed. Instead, we’ve introduced a workaround that allows compiler packages to be installed in a way that maintains their functionality.

The solution involves installing OCaml compiler packages to a global, per-user directory, rather than within the project’s sandbox. By default, the compiler is installed in a directory such as ~/.cache/dune/toolchains/ocaml-base-compiler.<version>-<hash>. This ensures that the compiler remains in its expected location and operates correctly without the need for relocation.

Moreover, this approach offers an added benefit: compilers installed through Dune can be shared across multiple projects. If two projects use the same version of the compiler, the installation step can be skipped in the second project, significantly speeding up the build process. Given that installing an OCaml compiler can take several minutes, this optimisation will save developers considerable time.

In summary, this new feature represents a substantial improvement to Dune’s Package Management system, making it easier and faster for developers to set up and manage their OCaml projects. By enabling direct installation of OCaml compilers through Dune, we’re removing a major barrier to adoption and enhancing the overall development experience.

Since it works transparently whenever you build a project with the Developer Preview, we'd love for you to test it out! Try out the new Dune Developer Preview today, and let us know how it goes on Discuss. We’re eager to see how the community leverages this new capability and look forward to your feedback as we continue to refine and expand Dune’s Package Management features.

The motivation behind Dune Package Management is clear. For years, the OCaml community has called for a single tool to address all the development concerns: building projects, managing dependencies, testing on different compiler versions, etc. By integrating package management directly into Dune, we want to resolve the above long-standing pain points that can make OCaml cumbersome to work with, both for newcomers and experienced developers.

The Vision for Dune

Our long-term goal is to make Dune the central tool for OCaml development. That means more than just feature additions! It's about radically simplifying how developers work with the OCaml platform. By making installation painless and simplifying frustrating workflows, such as the handling of dependencies and testing against multiple compiler versions, Dune will address all your OCaml needs.

Dune integrates package management by using opam as a libary in essential parts of our approach. Two commands lie at the heart of integration: dune pkg lock and dune build. dune pkg lock creates a generated lock file, whereas dune build depends on this lock file to manage project dependencies. You can now handle everything from project initialisation to dependency management using these simple commands.

What We’ve Achieved So Far

We've accomplished a lot in these past few months! The work we have done for Dune Package Management can already handle such complex projects as OCaml.org and Bonsai using the new package management features. Both were successfully built using these new features. These early successes confirm our hypothesis: we are on the right track, because this proves the solution's viability in real world scenarios.

But this is not the end of the work. In the future, we plan to further improve the UX so that Dune is not only correct but also easy and productive for developers to use. The remaining challenges are yet to be overcome, and we hope to make Dune Package Management the standard tooling for all OCaml workflows.

The Road Ahead

Now that we hit the milestone for MVP, the subsequent phase will have testing, validation, and enhancement of the developer experience. Our main focuses going ahead will include:

• Smoothen UX: We want to make the Dune Package Management interface as intuitive as possible, so developers can get their projects underway quickly.
• Optimising Performance: This means shorter compilation times, quicker install times for dependencies, and ensuring all operations work seamlessly.
• Simplify Tooling: We're starting to include things like testing, formatting, documentation generation, and more! This way developers will no longer have to run several different tools to manage their projects.
• Providing Clear Documentation: Thorough, user-friendly documentation will be essential in helping developers adopt these new features.

A Unified Future for OCaml

Package Management brings in a new era of OCaml development. Dune will now be the only tool engineers will need, making OCaml development as seamless and effective for both complete beginners and experienced developers on the platform.

We look forward to the future and what Dune Package Management will facilitate within the OCaml community. Stay tuned, and prepare to take part in a more integrated and seamless OCaml development experience with Dune.

Why the Wait?

Our progress has been slower than anticipated because the Dune project is a complex endeavour. We’re committed to balancing support for existing OCaml users while introducing new workflows. While opam remains the stable workflow and isn't going away, a key difference is that the Dune Developer Preview showcases experimental workflows. We believe, once fully ready, this will be transformative for the community. Meanwhile, we continue to maintain opam and its infrastructure, ensuring the OCaml ecosystem remains robust, with recent updates like opam 2.2 bringing proper Windows support.

Why the Change?

We have listened over time to your feedback and learned that while functional, the existing tooling around OCaml can be quite cumbersome. Our mission is to make OCaml development easier, faster and more enjoyable, and Dune is now about to become that all-in-one tool which does just that. This separation will allow Dune to cut down on the complexity and make its development process much lighter.

The Developer Preview gives a sneak peek into these improvements and provides an opportunity for developers to get their hands on these tools before the official release.

What's New in Dune?

The Dune Developer Preview introduces several major changes in how OCaml developers will work. Probably the most notable change is that it includes package management right within Dune itself. No more juggling with multiple tools; Dune does it all in one go.

Accordingly, the new Dune comes as a binary. That means it no longer requires the usage of opam to install it. You can simply just install Dune within minutes, and you'll be off to a flying start. That gives you so much more time for actual development.

Drawing Inspiration from the Best

Throughout this process, we've drawn inspiration from other successful ecosystems. We learned from Rust, Go, and Erlang the power of letting developers test and provide feedback on a pre-general release, and we are doing the same with Dune: giving the OCaml community a chance to have their say in shaping up the final form of this tool. Your input will be invaluable in refining Dune and ensuring it meets your needs.

Join the Beta

We are looking for developers to participate in the Dune Developer Preview. Experienced developers can help us put the latest features of Dune through their paces so that we can fine-tune the workflow to be smooth and intuitive across a range of projects. Those new to OCaml can test the workflow from their perspective of working in other ecosystems. Whether you have been using OCaml for years or are new to the OCaml ecosystem, your input is crucial in helping us make Dune the recommended tool for OCaml development.

To become a beta tester, one only needs to download the Dune Developer Preview. You will get the very latest version and immediately start playing with the new workflows and package management features. Your feedback will help to further shape the future of OCaml tooling, and we want to hear your thoughts on everything from usability to performance enhancements.

Measuring Success

Our goal is clear: use OCaml with as little ceremony as possible, so we set a few key benchmarks to make that happen. First: via Dune, the user should be able to fire up a new project in less than 10 seconds with no extra tooling required. This level of speed makes all the difference during everyday hacking.

We also monitor general developer satisfaction with the new Dune workflows. For the Net Promoter Score, we will know how likely a user is to recommend Dune to other users. Our objective is to reach +80, which shows great approval and satisfaction within the community.

What's Next?

This is only the beginning of the Dune Developer Preview. We will regularly update and enhance it based on feedback from beta testers. This includes polishing the new Dune Toolchain feature in order to automatically manage OCaml versions and development tools, and exploring new distribution channels in order for Dune to be even easier to download and use.

This is huge for OCaml, and we are excited to have you along for the ride. Why wait? Download the Dune Developer Preview today, and give some of the new workflows a try before letting us know what you think. With your help, we can make Dune the go-to tool for all OCaml development.

We can't wait to hear from you — and happy coding!

At Tarides, we are launching a new initiative to share our expertise and industry experience in a series of customisable, flexible training courses designed to unlock new possibilities for your teams.

How Tarides Can Help You

Technical training equips your teams with essential skills to handle the latest software advancements and workflow improvements. While many generic courses exist, specialised knowledge is often necessary, and our targeted training suite offers:

• Expert Instructors: Learn from core developers with years of industry experience
• Hands-On Learning: Jump straight into coding on day one, with exercises and real-world scenarios
• Peer Collaboration: Share knowledge and insights with a community of like-minded developers
• Career Advancement: Acquire new skills to face new challenges
• Flexibility & Customisability: Choose your custom topic elements to ensure the course works best for your needs

Our bespoke training can be customised to the unique circumstances of each team, addressing weaknesses and enhancing strengths. Each course offers:

• Flexible sessions: On-site, online, or in a hybrid configuration to fit your schedule
• 10 hours of post-training support in every package to address follow-up questions

Sign up online for our courses, and we will contact you for an initial consultation to provide more information, discuss options, and help you decide if training is right for you!

Our Courses

We have created a group of courses tailored to different needs, ranging from onboarding a team with OCaml to mastering specialised skills, from basic compliance with cybersecurity guidelines to auditing custom workflows. While they address various applications, all of our training shares a common goal: to minimise friction for busy organisations. Our most popular courses are detailed below:

Getting Started With OCaml: An Introduction

Are you using OCaml for the first time or onboarding new team members? This course is a perfect fit. It covers the fundamental language concepts, tools, and techniques and culminates with a practical exercise in which participants build their own application. The course includes modules on the Dune build system, the OCaml Platform, imperative and modular programming, debugging, and more!

OCaml stands out among its peers with its expressive syntax, robust type system, and exceptional performance. After this course, your team will walk away with an understanding of OCaml's main features and how to start using them to their advantage.

Mastering OCaml: Advanced Techniques

This is the best choice for teams already familiar with OCaml and who are using it in their projects. It enables your teams to adopt advanced techniques, such as Web application development with JSOO and WSOO, multicore programming with Eio, MirageOS, testing, and GADTs, making expert-level techniques accessible to our clients.

This course allows you to customise modules to cover the exact tools and techniques your teams need the most. This allows them to improve the quality of the code they produce precisely and effectively, boosting OCaml developers’ confidence and skills.

Scalable, Flexible, and Powerful: Language-neutral Functional Programming

Help your teams understand the core functional programming principles that help developers produce safer, less buggy, and more readable code - regardless of their programming language. This three day course is a rich introduction to functional programming. The first day focusses on the foundations, including recursive and higher-order functions, type annotations and type inference, and function composition and pipes. The second day delves deeper into types covering topics like immutability, monads, and currying. The final day pushes further into I/O monads, continuations, type algebra, and more.

This course is the perfect choice for teams responding to the growing push for safer code, wanting to adopt functional programming in their workflows. It is also an excellent choice for onboarding new teams, helping them gain confidence and competence with a new way of programming.

Coming soon: More Courses and More Content!

Some of our training programmes are still under development. Register your interest in the upcoming courses to find out when they become available.

• Open-source Development: How to make OSS work for you: Introduces the methodologies and best practices of open-source development, including how multiple contributors improve the quality of the end product, innovation of the ecosystem, and lower the overall cost of development.

• Cybersecurity & Secure-by-Design: Teaches the fundamentals of secure-by-design principles, how programming language design affects security, and common attacks and their mitigation. Furthermore, for EU-based clients, we will include how to comply with the Cyber Resilience Act. We will also offer an optional additional audit of your systems' vulnerabilities and suggestions for improving their resilience.

Get in Touch

We're excited to provide this service to our clients and know how important it is to share the knowledge we have accumulated. As the world increasingly recognises the need for change in how software is developed, OCaml, functional programming, and open source are growing. We are here to help you be ready for this transition!

Stay in touch with us on Bluesky, Mastodon, Threads, and LinkedIn. We look forward to hearing from you!

For a quick introduction to OCaml, check out the tutorials on OCaml.org.

What is Dune?

Dune is much more than a simple build tool. It automatically compiles your OCaml code, manages its dependencies, and generates the final executable or library. It's also a well-maintained, highly-optimised platform that streamlines your development process. Spend more time writing code and less on struggling with complex build rules.

Advantages of Dune

1. Consistency Across Projects

With Dune, you can be sure that the build processes are consistent, no matter how many different projects you are managing. This is very helpful when collaborating with other developers or when maintaining multiple projects. Once you work with Dune on one project, it's easier to work on the next, even if it has a totally different codebase, because Dune standardises how things are done.

2. Integration With the OCaml Platform

Dune lives on the cutting edge of the OCaml Platform (a set of tools and libraries) and forms a solid foundation for your development environment. Dune automatically plays nicely with other tools such as opam (an OCaml package manager) and helps you manage dependencies, run tests, and set up project documentation.

3. Performance Optimisation

Dune is fast and efficient. It tracks dependencies and rebuilds only when necessary, so your development processes will be more responsive. This performance optimisation benefits the developer, regardless of project size. Although for big projects, it especially makes a difference because it significantly reduces the build time.

Dune also supports other languages and tools within the same project. This flexibility makes it easy to incorporate C stubs, inline assembly, or even JavaScript (via js_of_ocaml and Melange) into your OCaml projects without needing to change your build system.

A Well-Maintained and Evolving Tool

The Dune team listens to community needs and regularly releases updates for performance, features, and bug fixes. This keeps Dune current with OCaml's development, giving engineers a coherent and state-of-the-art tool that evolves with the language and ecosystem.

Soon, Dune will also provide package managing functionality, so you can choose whether to use Dune or opam. It's currently in beta testing, so watch this blog for an announcement of the upcoming release!

Getting Started With Dune

It is very easy to install Dune using opam:

opam install dune

Now make a new OCaml project by running:

dune init project my_project

This creates a minimal project structure with sensible defaults, so you can get to coding right away. When ready, compile your project using dune build and run it using dune exec ./my_project.

Conclusion

Dune simplifies the development process, ensures uniformity, and is deeply integrated with the entire OCaml Platform. If you set up Dune from the very beginning, it will let you focus on creating great software, the most important thing.

As you learn more about OCaml, you'll appreciate the power, flexibility, and great community behind Dune and OCaml as a whole. For both small personal projects and collaborations on big applications, Dune is a tool you can rely on from start to finish.

Let's take a look at what the interns have been up to over the past six months. Remember, if you want to intern with us, keep an eye on our careers page for upcoming opportunities!

Eutro and Olly

@eutro's internship focussed on making the observability tool Olly more useful for developers by addressing two shortcomings: an incompatibility between Olly and the Runtime Events API that would cause crashes in certain cases, and the number of Olly's dependencies which made it difficult or impossible to use its core functions with unreleased ("trunk") OCaml (or at all on Windows). To resolve these issues, @eutro modularised Olly, implementing a table-based translation of runtime event names and tags.

@eutro modularised Olly by refactoring the binary into smaller libraries, with the awkward dependencies isolated into an optional library. Splitting up the libraries gives users greater control over their dependencies and you can learn more about modularisation in the PR.

The second part of the internship focussed on the table-based translation of runtime event names and tags to allow different versions of OCaml to consume runtime events. The goal was to avoid two bugs that arose when Olly profiles a program compiled with a different version of OCaml, olly trace generating nonsensical names for the slices and olly gc-stats silently generating garbage output. To delve into the details, check out @eutro's PR about runtime event names.

Lastly, @eutro managed to squeeze in some bug-fixes as well! One PR addresses the incorrect use of snprintf_os in formatting the runtime events ring file path and another that fixes some memory bugs in runtime_events_consumer.c. Both of these fixes will be included in the 5.3 OCaml update.

Lee Koon Wen and Eio

The Eio library enables users to write high-performance I/O programs leveraging the new effects system that came with OCaml 5. The I/O library pairs well with the io_uring interface on Linux as a rule; however, the asynchronous nature of io_uring can make it hard for the developer to get a grasp on the performance of their Eio programs when they are bottlenecked by something within the kernel rather than the program itself.

The Linux kernel has a mature and extensive system for gathering data on its performance. In his internship project, Lee Koon Wen's task was to produce a library that used the Linux eBPF probes to gather kernel-side data on an Eio program's io_uring use and deliver them to the program's runtime events. This would let users analyse kernel-side performance data alongside their program's performance using an observability tool like Olly.

Two projects have sprung from this work, uring-trace and ocaml-libbpf. The former is a tracer that, using bindings provided by the latter, can extract events from a Linux kernel. These traces can then be generated in Fuschia format and displayed on Perfetto. This project will benefit developers on the Linux platform, helping them understand and optimise their programs using accurate data.

Ryan and Polyglot Package Management

Using several programming languages in one project lets you take advantage of the particular strengths of each language and of its library ecosystem. For example, Python is well-known for its data science libraries, Rust for its ownership memory model, and OCaml for its type safety. Over the past couple of decades, many large programming language ecosystems (and even some smaller ones) have acquired language-specific package managers, e.g. pip (Python's; 2008), cargo (Rust's; 2015 - although it started with one) and, of course, opam (OCaml's; 2013). Managing these so-called 'polyglot programming' projects, with several languages working together, relies on coordinating these package managers to provide language libraries and toolchains like compilers and build systems. The need to use multiple package managers naturally increases the complexity of these projects. Additionally, dependencies are hard, or impossible, to express across different package managers.

Ryan Gibb's research internship at Tarides focussed on using nix as an initial bridge towards these objectives, extending opam to support the provision of "external dependencies" using Nix, the language-agnostic functional package manager, instead of the OS's own package manager. There has been much work in this area already (e.g. opam-nix and opam2nix), but these have focussed more on being able to take opam packages themselves and install them using nix.

Ryan's work moves in the other direction, allowing Nix packages to be used within the environment set-up by opam, by adding a depext mechanism to opam. Parallel with this work, Ryan also extended a previous investigation with nixpkgs to allow users to be able to specify versions of Nix dependencies. In general, Nix only supports the latest versions, but by analysing nixpkgs repository history we can map the versions of the packages we’re interested in to the ranges in nixpkgs commit history which provide them. Using opam’s solver we can then find the maximum commit of nixpkgs satisfying the version constraints on the packages (as long as a state exists meeting the conditions).

Future work will address current limitations and bring improvements to the workflow for users. For example, opam’s Nix depext mechanism picks up the environment variables from the builder's shell, meaning it must manually specify the environment it wants to extract. It may be possible to access the env attribute of derivations directly as the Nix binary does. Ryan intends to keep working on these limitations as well as future goals, including shepherding Opam's Nix depext support through review and possibly productionising opam-nix-repository for use in opam-repository.

For more information, check out the project's PRs, #5982 and #2.

Nikolaus, Gospel and Ortac

The culture around OCaml values safety and reliability, so it is no surprise that a suite of tools has been developed to ensure these qualities. One such tool, Gospel, is a contract-based behavioural specification language that can provide a logical model for OCaml types and describe the intended behaviour of functions using pre- and post-conditions. Ortac, in turn, is a tool that can generate a QCheck-STM test suite based on the Gospel specification of a library. You can find out more about them in the dedicated post Getting Specific: Announcing the Gospel and Ortac Projects.

The goal of Nikolaus Huber's internship was to lift some of the limitations of Ortac and QCheck-STM to expand its use cases for developers. It's essential to increase the types of tests that users can generate so that more OCaml code can be checked using Gospel. His project has produced three PRs, #235 which centres on allowing tests to run without system under test in the signature, #237 focusses on adding support for tests with tuples in their signature, and #247 aiming to introduce support for testing functions with multiple systems under test as arguments.

In addition to the PRs addressing Ortac limitations, Nikolaus has also fixed several issues regarding Gospel and Ortac. He added a new error for when there are no commands produced during the translation from Gospel to OCaml, fixed a bug within the QCheck-STM that occurred when testing functions that return integers, and addressed another bug where a type check would incorrectly indicate that code was correct. All in all, Nikolaus' project benefits developers who want to test their OCaml programs to ensure they perform predictably and correctly.

Stay in Touch

We want to hear from you! Follow us on Bluesky, Mastodon, Threads, and LinkedIn for the latest news from Tarides and to share your thoughts with us. Are you interested in completing an internship project with us? Keep an eye on our careers page, where we announce upcoming internship opportunities. Happy hacking!

If you want to know what programming with Eio is like from another user's perspective, this interview is for you. Let's explore what Simon had to say!

The Interview

"Hello Simon, thank you for meeting with me today! What first got you curious about Eio?"

“We use OCaml at Asemio, and I use OCaml a lot for my own projects. But when it came to concurrent code, I had accepted that to reap the benefits, I needed to deal with the increased complexity that came with them. In traditional concurrency, the developer relies on abstractions for tasks and promises using async/await. I had grown to assume that coloured functions was a necessary price to pay when the Eio library with its effect handlers announced that we didn't have to deal with this problem at all."

"Concurrency squeezes more out of code, so people use it, accepting that they will pay the cost of increased complexity. But we can enjoy the benefits of concurrency without the cost!"

"I had heard of Eio relatively early in its development, around version 0.3. At that point, it was still in flux, and I wasn't comfortable that I could build something lasting on top. Then I saw the announcement post for version 0.10 on June 11th 2023, and that's when I decided to try and switch. The project looked much more stable, the developers had resolved several initial issues, and the code wasn't being rewritten nearly as much as before. All of my previous hesitations had been addressed.”

"What did you do next?"

“I had a project in mind for testing Eio, a tool I had initially developed as an internal application. At Asemio, we regularly process huge Excel files (think millions of cells). The challenge with these files is that there are multiple ways that they can be organised internally, and it is impossible to know how they are organised without opening them. We need to be able to stream and read the data of very large files.

This is where the SZXX library comes in. It's a streaming ZIP, XML, and XLSX library that can stream data from these file formats even when reading from a network socket, either in constant memory or with user-defined memory usage guarantees. I had initially used the Lwt library to implement concurrency in SZXX, but I decided to convert it to Eio.”

"What benefit did you expect to see from switching to Eio?"

“From my perspective, the main problem with using Lwt was the function colouring problem. Micromanaging code in that way is time-consuming, and I wanted to learn Eio mainly because I knew that effects could potentially eliminate the extra work and complexity that came with Lwt.

I also learned that Eio came with integrated access to io_uring, letting the developer use it without additional effort and the function colouring problem. These were the two features I was the most interested in when converting the SZXX project from Lwt to Eio.

In addition, I had observed a split between Lwt and Async within the community, and for a while, I had been looking for a solution that could bridge the gap. From my research into the I/O library at that time, I felt confident that it had the potential to 'heal the split' between the different library users. In a way, converting SZXX to Eio was my way of testing that theory to see what effect the transition would have."

"What was your initial experience with Eio? Did you have any frustrations?"

"I actually did! When I first started using Eio, I had a hard time understanding how Paths, Flows, Files (and others) relate to one another. I also experienced a lack of documentation and examples of abstraction to help the newcomer get up and running. These are some of the first things a new user will experiment with, and they must work well. Since then, I have helped improve the documentation (alongside several other contributors) and made other quality-of-life contributions to the library."

These initial friction points were not enough to dissuade me from continuing my project, and after that initial hurdle, I was very happy."

"I had been prepared to accept a performance degradation to get the benefits I was looking for, but I was pleasantly surprised to note that performance actually improved.”

"That's great! Did Eio affect your project in any other ways? Did anything surprise you?"

“I was unprepared for just how much of a difference using non-monadic code would make. Eio has this concept called a Switch, which ties resources to scopes. It cleans up your code by managing the lifecycle of open resources automatically, for example, turning off background processes, closing file handles, and ensuring that all auxiliary states are set to their desired state upon exiting the scope.

When you write parallel programs, there can be hundreds of thousands of interleavings and computations, and ensuring that each computation is handled correctly has always been a challenge with monadic-style code. But with Eio and effects-based concurrency, the switch handles much of that complexity for you.

In fact, both of these benefits work synergistically. They include solving the function colouring problem (meaning you don't need to distinguish between async and 'normal' functions) and using non-monadic code. Each amplifies the other almost exponentially and frees up a lot of your complexity budget."

"The human brain limits the complexity of code, and at some point, you just can't keep making things smaller to make them simple enough. Eio helps reduce complexity. With Eio, I was able to simplify and remove so much bloat from the code and achieve some really tricky optimisations, pushing the limit of what was possible.”

"That's an interesting insight, and I think one that only someone with real user experience can make. It's great to hear what results the user might expect. I know you have made significant contributions to Eio as well, and that the team has made you a maintainer in response to your contributions and reviews. Can you tell me about some of those?"

“I started contributing to Eio from around version 0.11 and onwards. Some of my main projects include: executor_pool, developing an Eio adapter for the popular CSV library that I also added to Angstrom, as well as work on an internal thread pool to speed up platforms without io_uring support. Along the way, I have added resources to Eio that make it easier for people to adopt the tool. For example, I designed the executor_pool module based on common patterns that I had often used in my projects.

Regarding the adaptors, in OCaml, library authors need to explicitly write one adaptor per I/O library they choose to support. The popular CSV library already had adaptors for both Async and Lwt. I knew that it would be hard for certain users to pick up Eio if it did not have an adaptor, so I wrote one and added it to both CSV and Angstrom. A lot of OCaml software depends on Angstrom through one dependency or another, and without Eio support, those codebases cannot make the switch to Eio.”

"Thank you for answering my questions and sharing your experience with Eio! Do you have any final thoughts you would like to share?"

“Eio helped me reason about my code, and I discovered bugs and problems because of how much Eio had cleaned up the code. I uncovered hidden bugs in every program I converted from Lwt to Eio. Every single one also ended up being faster, not because Eio itself was faster (it was as fast as Lwt), but because of the optimisations I could now afford to make, thanks to the reduced complexity.”

Stay in Touch

If you have questions about Eio or how to use it, you can fill in our contact form, and we will contact you. You can also ask questions on the OCaml Discuss forum or open issues and pull requests in the Eio repo.

To keep up with our activities, you can always follow us on Bluesky, LinkedIn, Mastodon, Threads. We look forward to connecting with you!

What is odoc?

odoc is a powerful documentation generator specifically designed for the OCaml programming language. It transforms OCaml interfaces, libraries, and packages into clean, readable HTML, LaTeX, or man pages. If you've worked with JavaDoc or Doxygen in other programming languages, you'll find odoc to be a similarly indispensable tool in the OCaml world.

The purpose of odoc is twofold:

1. It helps developers create comprehensive documentation for their projects.
2. It allows users to easily navigate and understand these projects through a standardised format.

As OCaml projects grow in complexity, well-maintained documentation becomes increasingly important for collaboration, onboarding new team members, and ensuring long-term project sustainability.

The odoc Cheatsheet: A Quick Reference for OCaml Developers

While odoc is great for generating docs, it uses a syntax that is not widely known. Learning a new syntax can be cumbersome, if not downright difficult. Before this cheatsheet, the resource for the syntax was only in the odoc for Authors page. However, this page offers extensive detail, covering far more than just the syntax. While excellent for in-depth exploration, it can be challenging when you're aiming for quick productivity.

The odoc Cheatsheet is a very simple resource for writing simple things. It is easy to read it and discover syntax, and you can use it to recheck your syntax. Rather than explaining, it provides examples, which is less cognitive overhead for the developer. It serves as a concise reference guide that covers the most important aspects of odoc, helping you to quickly get up to speed without wading through extensive documentation.

Here’s a closer look at how this cheatsheet benefits you:

1. Easy Access to Essential odoc Syntax The cheatsheet provides a useful list of odoc syntax. Whether you need to generate documentation for a single module or an entire project, the cheatsheet lays out the exact markup commands you need. This can save a lot of time, as you won’t have to search through various resources to find the correct syntax or options.

2. Concise and Well-Organised Information Information is presented in a clear, concise table that allows you to quickly find what you need.

This organisation is particularly beneficial when you’re in the middle of coding and need to find a markup command quickly. The cheatsheet gives you instant access to the most relevant information.

3. A Great Learning Tool for New OCaml Developers For those new to OCaml, the odoc Cheatsheet doubles as a learning tool. By following the syntax provided, you’ll not only generate better documentation but also gain a deeper understanding of how to structure your code and its corresponding documentation effectively.

The cheatsheet explains how to use specific annotations in your comments to generate informative documentation. This might not be immediately obvious to someone new to OCaml or odoc, but it can greatly enhance the usability of your generated docs.

Conclusion: A Simple and Useful Resource for odoc

Whether you're maintaining a large OCaml project or just starting out, the odoc Cheatsheet simplifies the documentation process, making it easier to produce high-quality docs with minimal hassle. Keep this cheatsheet at your fingertips, and ensure your OCaml projects are documented as well as they are coded.

So, before you dive into your next OCaml project or documentation task, take a moment to explore the odoc Cheatsheet. It could be the key to making your work more efficient and your documentation more effective.

This is part one of our feature parity series highlighting features returning to OCaml in an effort to restore feature parity with OCaml 4.14. When OCaml gained multicore support (that is, the ability to execute on multiple domains) it had far-reaching implications on the way the runtime worked, and as a result, support for some features were dropped for the 5.0 release. To address these gaps, a significant amount of work has been done behind the scenes to adapt tools and runtime features to work safely and performantly with multiple domains. Tarides is part of the effort to restore these familiar features to OCaml 5.

What is Compaction?

In OCaml 5.0, the major heap in the parallel Garbage Collector employs size-segregated pools attached to each domain. Over time, as OCaml domains allocate and discard heap values, many of these pools end up being only partially used. For example, a program might allocate millions of two-element tuples when initialising but no longer need most of these afterwards. This will result in lots of 'size two' pools, most of which will only be sparsely filled. This is inefficient as OCaml will still consume system memory for all the pools, even if the heap values only take up a tiny proportion of each pool.

Compaction is not new to OCaml; the latest version to include the technique was 4.14. After OCaml adopted a multicore garbage collector, compaction needed to be rewritten to work with the major heap's new structure and to be safe in parallel execution. Compaction for OCaml 5 - as reintroduced in PR #12193 - achieved this by identifying a small number of shared pools in each size class big enough to contain all heap values in that size and then moving all heap values in the remaining pools into the selected pools. This results in many empty pools, the memory of which can be returned to the operating system.

The new algorithm is entirely different from the previous one, so let's look at how it works!

Compaction in 5.2

Allocating Into the Major Heap

Simply put, compaction is a means of rearranging fragmented pieces of memory to larger - compact - chunks. To understand how the compaction introduced in 5.2 works, we must first understand how allocation works in the GC's major heap (you may want to skip this part if you're already familiar with the process). When you first allocate a value in the major heap, it is given a size from a size class table. Each size class, in turn, lists four different ways a pool can be stored: unswept available, unswept full, available, and full.

These pools are divided into blocks, and each pool contains a number of blocks of the same size class. When we allocate a value, an appropriate size class is chosen depending on the size of the value, and the first pool with space available in that class is selected. The header of that pool has a pointer indicating the next available block, and the value is written into that memory block. Each domain is responsible for this process independent of other domains. This is crucial for acceptable performance in parallel programming.

When the GC sweeps the pools, it will free certain blocks and add them to the free list in the header of their pool. After a while, cycles of sweeping and allocation create pools with free ('empty') blocks interspersed among live ('full') blocks. This process results in inefficient memory use and many partially filled pools.

Compacting the Major Heap

To address this inefficiency, the developer can compact the major heap and move the live blocks into an optimised order among the pools. In OCaml 5, this technique follows a specific sequence, which is as follows:

1. Barrier:
Because this is parallel compaction, we must synchronise all the domains before proceeding. Each domain has its own heap, and the heaps are compacted in parallel, with each domain responsible for its own compaction. Synchronisation is achieved with the help of a barrier.

2. Size Class:
The compaction process iterates through each size class, processing one at a time, starting with the smallest. A stats table is allocated for each domain, with a slot available for each pool of the current size class being processed. Since the GC has already swept everything we will be compacting, there are only two states a pool can be in, full or available, where the latter means there is free space available in it.

The process then continues by using the stats table to check whether pools are full or available (meaning they have at least one free block). This process means we don't delve deeply into the memory to read from it, and there is no cache contamination. There is no synchronisation between domains in this step, and the compaction process for a domain only proceeds from here if there is at least one available pool.

3. Using the Stats Table:
By this time, each domain to be compacted will have a stats table with a list of all the available pools. In the next step, the process goes down each pool on the available list and counts the number of live and free blocks. This is done linearly through the pool.

Once the number of live and free blocks is known, the number of live blocks is deducted from the number of free blocks. The resulting number of free blocks lets us calculate which pools can be emptied and which will be retained. This is all the information we need from the stats table, so once this step is completed, the stats table is cleared.

4. Pool Pointers and Live Links:
To summarise, we now know how much live space there is within the pools and how much free space we can liberate if we compact the live blocks together. To achieve compaction, we create pointers to two pools, one to the first pool we are evacuating and one to the first pool we are retaining (for those who are curious, these pointers are named current_pool and to_pool respectively).

The process starts with the first pool we know will be evacuated. It finds the first live block within that pool and uses the current_pool pointer to remove it from the pool and the to_pool pointer to insert it into one of the pools we know will still be live post-compaction (this information comes from the calculation we did using the stats table).

5. Compaction!:
This is the operative part of compaction: copying all the live blocks from pools that will be evacuated into pools that will remain live after compaction using the two pointers. As this is done, the process writes forward pointers that point from the block where something used to be stored forward to the block where it is now stored post-compaction.

6. Barrier 2:
Again, another barrier syncs all the domains – a crucial part of compaction on multiple domains.

7. Scanning:
This part of compaction is the most expensive in terms of time. The entire OCaml heap has to be scanned for pointers pointing to old block locations (moved as the pool they were in was evacuated), and old pointers must be updated using the forward pointers. Each domain is responsible for updating its data.

This is a deceptively extensive process. For example, even pointers in objects that are too large for the size allocator (so over 128 words) and therefore never moved by compaction may still need to be updated after the compaction process as they may also contain pointers to the old block locations.

8. Barrier 3:
Another barrier synchronises between domains.

10. Freeing Evacuated Pools:
All the evacuated pools are freed and added to the free list.

11. Barrier 4:
Another barrier to synchronise between domains.

12. Release Memory:
One domain, whichever is the first one to get to that point, unmaps the free list. This means that the memory we asked the OS for initially, which currently belongs to the OCaml system, is released at this point and goes back to the OS. This is the end benefit of compaction; it reduces the size of the OCaml system on your machine and returns memory to the OS for use elsewhere.

Final Details & Next Steps

Before we wrap up, let's look at one more detail about how compaction works in 5.2. In 5.2, compaction uses a slab allocator and size classes, whereas OCaml 4 uses a free list. This means that OCaml 5.2 does not provide the option to set an allocation policy like OCaml 4.14. Our testing has found that for most workloads, the chosen allocation policy (using the size classes) performs well. However, expert users can tune the configuration of the size classes in gen_sizeclasses.ml (necessitating that they build their own OCaml), which they may find useful for their own projects. This is just one example of the challenge that comes with adapting a feature as complex as compaction to be compatible with multiple cores, and the careful weighing of pros and cons it requires on behalf of the developers.

The next steps for OCaml 5 are the expected restoration of MSVC backends, Statmemprof support, and the return of the unloadable runtime coming in releases 5.3 and 5.4. Keep a look out for future posts detailing those features and the efforts put toward bringing them back.

It's great to have compaction restored to OCaml, and it is a testament to the hard work of several teams within Tarides and the wider open-source community surrounding OCaml. We are happy to be part of the team working on this secure and performant programming language.

Share Your Experience!

We want to understand your experience! We're open to suggestions and feedback about the process to help us optimise the feature and deal with any pain points. You can share your thoughts on OCaml's discussion forum or make suggestions directly in the repo.

You can stay in touch with us on Bluesky, Mastodon, Threads, and LinkedIn. We look forward to hearing from you!

Tarides engineer Tim McGilchrist recently wrote a blog post that explores how to debug OCaml programs using LLDB on macOS. Developers familiar with languages like C, C++, or Rust may already have experience using LLDB, as it is a common choice on Linux or FreeBSD. LLDB is also the debugger that ships with XCode on macOS.

The Role of LLDB in OCaml Debugging

OCaml has traditionally used its built-in debugging tool ocamldebug for OCaml bytecode, but LLDB offers a way to debug native executables.

LLDB, the debugger from the LLVM project, offers a powerful way to inspect and debug compiled programs. While LLDB is not specific to OCaml, Tim's blog post highlights how it can be effectively used to debug OCaml code.

Tips and Tricks

Tim's post also provides practical tips for getting the most out of LLDB when working with OCaml. For instance, it discusses how to deal with OCaml’s optimised code, which can sometimes make debugging more challenging. It suggests compiling without certain optimisations when debugging complex issues, to ensure that the debugging information remains intact and the code paths are easier to follow.

Final Thoughts

Debugging is a critical skill for any developer, and mastering it in OCaml can significantly improve your productivity and code quality. The method outlined in Tim's post demystifies the process of using LLDB with OCaml, making it more accessible to those who are new to the language or who may be transitioning from other programming environments.

If you’re eager to dive deeper, please read Tim's full blog post, which gives both detailed instructions and examples to help you get started. With these tools at your disposal, debugging OCaml code becomes a much more manageable task. Happy coding!

The Gospel Project

The French National Research Agency (ANR) is a public institution in France that funds innovative research projects where public institutions collaborate with each other or the private sector. OCaml was invented at the French National Institute for Research in Digital Science and Technology, INRIA, and ANR is funding a research project as a collaboration between INRIA, Tarides, LMF UPSaclay, and Nomadic Labs. The goal of the project is to develop and improve the specification language Gospel alongside its tooling ecosystem and demonstrate its usefulness in different case studies.

What is Gospel?

Gospel is a contract-based behavioural specification language that allows you to write specifications in the module interface you want to specify. As a specification language, it is a formal language, meaning its semantics are precisely defined (by means of translation into Separation Logic, see this paper).

By behavioural specification language, we mean a language that allows you to describe the expected functional behaviour of a function. Specifications don't reference resources such as CPU time or memory size, but only what the program does (so-called functional behaviour). Expected behaviour is expressed as a contract. The basic premise is that as long as the user of the library calls functions with arguments that respect the expressed preconditions, then the implementation of the library should behave per their description.

Per se, Gospel doesn't guarantee that your implementation respects its given specifications, it is simply a language that allows you to express precisely what your code should do. However, note that Gospel still comes with a type-checker. This type-checker lets you check that your specifications are well-formed and in sync with the interface. For example, if you add an extra argument to a function in your library, the Gospel type-checker will tell you if you forgot to update the specifications accordingly.

Gospel is a relatively new specification language and is bound to evolve, but it is already mature enough to specify a diverse set of libraries. It provides developers with a non-invasive and easy-to-use syntax to annotate their module interfaces with formal contracts describing type invariants, mutability, function pre- and postconditions, exceptions, etc. For example, let's say you want to specify a fixed-size stack. The type declaration in the module interface would look like:

type a t
(*@ model capacity : integer
    mutable model contents : a sequence
    with s
    invariant Sequence.length s.contents <= s.capacity *)

You give two logical models to your datatype: an immutable one for the capacity of the stack and a mutable one for the content. Then, given a stack, s, you can formulate type invariants. Namely, the stack should not have more elements than capacity allows. The specification for the create function would look like:

val create : int -> 'a t
(*@ s = create n
    requires n > 0
    ensures s.capacity = n
    ensures s.contents = Sequence.empty *)

The first line binds the arguments and the returned value to names so that we can talk about them. Then we express the precondition that the given argument should be strictly positive and the two postconditions that fill the logical models as expected.

Gospel is also a tool-agnostic specification language, meaning it doesn’t make any assumption about how and by which tools its specifications will be consumed. Some users use Gospel specifications to provide proofs of functional correctness for their implementations. For example, Cameleer does so by leveraging the power of the Why3 deductive verification platform. At Tarides, with the Ortac project, we explore how to use Gospel specifications to do runtime assertion checking.

Gospel was initially developed by Cláudio Lourenço (LRI post-doctorate) and is currently maintained by Jean-Christophe Filliâtre, Samuel Hym, Nicolas Osborne, and Mário Pereira. Clément Pascutto also maintained Gospel for several years as part of his PhD work at Tarides and LRI.

A Tool for Gospel: Ortac

Ortac stands for OCaml RunTime Assertion Checking. Clément's PhD thesis initiated the Ortac project which has since grown into a greater cooperative effort. Samuel Hym and Nicolas Osborne currently maintain it. At its core, Ortac translates the computable subset of Gospel terms into OCaml code. In addition, it provides a plugin architecture to implement different uses of this translation. Translating Gospel terms into runnable OCaml code opens the possibility of checking an implementation against the interface specification at runtime.

Three plugins have been built upon this translation, plus a fourth one, which is slightly different. Let’s take a look:

1. The Ortac/Wrapper plugin was developed during Clément's PhD. Given the Gospel specified interface of a module, this plugin generates a new module with the same interface, wrapping the initial implementation with runtime checks coming from Gospel specifications. When a Gospel specification is violated by the client for preconditions or by the initial implementation for postconditions, the wrapped version will output an error message providing the user with useful information, such as which Gospel clause they have violated. Users can then use the new wrapped module in place of the original one in their project, to, for example, aid in debugging efforts. This plugin is still considered experimental.
2. The Ortac/Monolith plugin, which is based on Ortac/Wrapper and is the product of an internship, was presented at ICFP 2021. Given the specified interface of a module, this plugin generates the Monolith standalone program, testing the initial implementation against the wrapped one. The idea is that, in case the implementation doesn't respect the specification, the wrapped version will return a special Ortac error while the bare initial one won't. Monolith allows you to use fuzzing to test your library and provides a runnable scenario that demonstrates the unexpected behaviour. This plugin is also still considered experimental.
3. The Ortac/QCheck-STM plugin is based on Naomi Spargo's internship project. Given the specified interface of a module and some user-provided extra information, this plugin generates standalone QCheck-STM tests. In addition to avoiding having to write the QCheck-STM test by hand and as a recently added feature, in case of a test failure, the generated tests will inform you which part of the specification has been violated, give you a runnable scenario demonstrating the unexpected behaviour, and the expected returned value when the Gospel specifications allow to compute it. This plugin has been released.
4. The Ortac/Dune plugin is slightly different as it doesn't rely on a Gospel specification. Instead, it helps you by generating the Dune rules necessary to run another plugin. So far, only the command for Dune rules related to the Ortac/QCheck-STM plugin is available, as it is the only one that has been released. The command yields the Dune rules required to generate and run the tests.

Future Steps

Regarding Ortac, the 0.3.0 version of a set of Ortac packages, including the first Ortac/Dune release, has recently been published. Among other improvements and fixes, this release makes dynamic formal verification with Ortac very easy. The team is now investigating how to test more functions by lifting some limitations that come with a naive use of the QCheck-STM test framework, with an intern (Nikolaus Huber) working on this last topic.

With Gospel 0.3.0 published, the upcoming goals centre around continuing maintenance and development, using our engineering expertise to help our research partners bring new features to the Gospel language.

Until Next Time

Do you want to try out Gospel and Ortac? Check out the documentation and report back on your experience! If you want to stay informed about our projects, follow us on Bluesky and LinkedIn for all our latest updates. Are you interested in using Gospel for your own projects? Contact us, and we will be happy to discuss the benefits of implementing specification languages in your workflow.

Can't wait for September 2nd? Check out the talks we're bringing to the conference this year and get a taste of what's to come!

Tarides Talks

We are excited about all the talks at the OCaml Workshop this year, and it's great to see such a variety of topics being presented. You can browse all the accepted papers on the OCaml track of the ICFP website. This year, Tarides team members will give five talks at the OCaml workshop and one at the ML workshop, so let's dive into the topics!

Wasm_of_ocaml at the ML Workshop by Jérôme Vouillon

Jérôme Vouillon will be presenting on the fork of the well-loved Js_of_ocaml compiler, Wasm_of_ocaml, which translates OCaml bytecode to WebAssembly (Wasm) rather than to JavaScript. Wasm is a low-level virtual machine that is both platform- and language-independent and is attractive to developers as a portable compilation target deployable on a wide variety of platforms. Wasm_of_ocaml is already fairly mature, having been successfully used on large programs.

Curious about what this means for OCaml users? Jérôme's talk will give you an overview of the compiler's features, including that it's highly compatible with Js_of_ocaml (one can compile existing programs with minimal changes) and offers a performance boost over the same (2x or more!). In addition, the talk will give you some background to Wasm_of_ocaml's design, the implementation of its runtime environment, how it interacts with JavaScript APIs, how we can take advantage of JavaScript to implement functionalities that are currently unavailable in Wasm, and share some benchmarks to give you an idea of how the compiler performs. Check out Jérôme's talk in the ML track for all this and more!

First-Class Windows: Building a Roadmap for OCaml on Windows by Sudha Parimala, Benjamin Canou, Pierre Boutillier, and David Allsopp

The goal of the First-Class Windows team (which we discuss more in our blog post introducing the project is to bring support for the Windows platform up-to-par with Tier-1 platforms like Linux and macOS. Reaching this milestone will require planning, collaboration, and iterative change. Still, we expect the process will significantly enhance the OCaml experience for many users. Check out the talk at the OCaml Workshop track at ICFP to learn all about the project's background, the current state of OCaml on Windows, the launch of the Windows Working Group, what information they are gathering, and where the project is heading.

Most importantly, the talk will unveil the new roadmap, which outlines the plan for addressing the pain points and improving OCaml on Windows. The roadmap is intended as a living, collaborative document. Join the talk to be part of the discussion and improve OCaml on Windows!

Project-Wide Occurrences for OCaml: A Progress Report by Ulysse Gérard

Our next talk will be great news for anyone who writes programs for OCaml! Good editor tooling is indispensable when writing programs, and different language servers provide a variety of features.

Before the OCaml 5.2 update, the main language servers were limited to providing occurrences inside the active buffer. After the update with the latest Dune, Merlin, and ocaml-lsp servers, users can take advantage of a new feature enabling project-wide usages search. It allows users to list all the occurrences of a given value, type, or module, quickly navigating between each instance.

The team prioritised three key areas when developing the feature: correctness, exhaustivity, and performance. Correctness refers to the tool's ability to list the occurrences of a value without including false positives; exhaustivity refers to the fact that it needs to list all occurrences and perform the way it yields these results in a reasonable amount of time.

Come to the talk for a demonstration of the powerful new search features available in OCaml! Learn about the project's design and implementation, the challenges they encountered (including some noteworthy patches to the compiler, Dune, Merlin, and ocaml-lsp!), and improvements planned for the future.

Picos – Interoperable Effects-Based Concurrency by Vesa Karvonen

Picos is an ongoing project created to allow users to mix and match effects-based concurrent programming libraries and async I/Os. OCaml 5 introduced support for parallelism and effects, which enables the implementation of direct-style cooperative concurrency libraries like Eio. In fact, many such libraries have been created, including Moonpool, Domainslib, Miou, and more!

The current difficulty is that all of these libraries are incompatible, meaning that a program that uses one cannot directly use another. This also means that, theoretically, libraries would need to provide an ever-increasing number of backends to ensure their users could combine them with the effects-based scheduler of their choice. This constant game of catch-up isn't desirable for developers or users of these libraries, and Picos aims to solve this problem.

Picos is an interface that enables interoperability between different libraries, created to provide users with greater flexibility and choice. Vesa's talk will introduce the nature of the problem, technical details of how Picos is implemented, details about its performance, and a vision for the future of schedulers in OCaml with Picos.

Opam 2.2 and Beyond by Raja Boujbel, Kate Deplaix, and David Allsopp

The hardworking team behind opam 2.2 are excited to present the update and share the new features! The update's main feature is native Windows support, something many community members have been looking forward to. With the project commencing in 2014, it has taken a significant amount of work and a long time to reach this important milestone – a process you can learn all about in their upcoming talk!

The talk will give listeners an opportunity to understand all the different elements that came together to bring native Windows support to OCaml, along with some of the other new features added in opam 2.2. This includes often overlooked aspects of releases like bug fixes and functional testing for stability.

Additionally, the talk presents insights into the maintenance of opam and moving towards a new release cycle, hinting at what the future will bring. If you're at all curious about opam and how maintainers bring new features to users, this is the talk for you!

Saturn: A Library of Verified Concurrent Data Structures for OCaml 5 by Clément Allain, Vesa Karvonen, Carine Morel

This talk presents a useful new OCaml 5 library that offers a collection of ready-made efficient concurrent data structures that are well-tested, benchmarked, and, in part, formally verified.

Parallel programs are complex, and sharing data between multiple threads presents a well-known difficulty. Using locks is a known way of managing this complexity, but it is not always the best option, potentially introducing unsatisfactory performance and liveness issues. In such cases, lock-free implementations may be preferable, but their complexity can make them hard to design. Saturn saves the OCaml 5 developer the trouble of designing their own lock-based or lock-free data structures by providing them with a selection of standard ready-to-use structures.

Join the talk to hear all about the library's design and details about the benchmarks, tests, and formal verification the team has done. You can look forward to a technical deep dive full of details about what creating parallel programs in OCaml really involves!

Until Next Time!

If you're attending ICFP this year, come and find us! We would love to chat with you about everything OCaml and functional programming. To stay up-to-date, you should follow us on LinkedIn and Bluesky. We look forward to hearing from you!

The traditional "occurrences" query in Merlin modes, named "Find All References" in LSP-based mode, was used to only return results in the active buffer. This is no longer the case!

Occurrences queries will now return every usage of the selected identifier across all of the project's source files.

There are some limitations that come with this initial release. When queried from an identifier's usage or its definition, all other usages of it are returned, but related declarations are not. In particular, this means that queries should be made from implementation files, not interfaces (.mli).

In this post, we will give an overview of the ecosystem's various parts that need to work together for this feature to work.

Try It!

Before diving into technical details, let's see how it works. You can try it on any project that builds with Dune and is compatible with OCaml 5.2.

Update your switch by running opam update && opam upgrade to get the required tool versions:

• Dune >= 3.16.0
• Merlin >= 5.1-502
• OCam-LSP >= 1.19.0

Since we are looking for all occurrences, we need to build an index for Merlin and LSP. Fortunately, this is well integrated in Dune, and you can build the index for your project by running:

dune build @ocaml-index

This alias ensures that all the artifacts needed by Merlin are built. You can also add --watch to always keep the configuration and the indexes up to date while you edit your source files.

Note that unlike dune build @check, the @ocaml-index will build the entire project, including tests.

Once the index is ready, you can query for project-wide occurrences:

• merlin-project-occurrences in Emacs
• MerlinOccurrencesProjectWide in Vim
• Find All References in LSP-based plugins.

Here is a comparison of a references query before, and after building the index:

Now, let's dive into more technical details.

High-Level Overview

The base design is fairly simple. We want to iterate on every identifier of every source file, determine their definition, and group together those that share the same definition. This forms an index. Tools can then query that index to get the location list of identifiers that share the same definition.

The following section describes how we implemented that workflow:

1. Compute definitions using two-step shape reduction
2. Driving of the indexer tool by the build system
3. Changes to Merlin to properly answer queries

Two-Step Shape Reduction

Finding an identifier's definition in OCaml is a difficult problem, mostly because of its powerful module system. A solution to this problem has been recently described in a presentation at the ML Workshop: shapes. In short, shapes are terms of a simple lambda-calculus that represent an abstraction of the module system. To find an identifier's definition, one can build a shape from its path and reduce (as in beta-reduction) that shape. The result should be a leaf with a UID that uniquely represents the definition.

This has been implemented in the compiler, and Merlin already takes advantage of it to provide a precise jump-to-definition feature.

For project-wide occurrences, we perform shape reduction in two steps:

First, at the end of a module's compilation, the compiler iterates on the Typedtree and locally reduces every identifier's shape. If the identifier is locally (in the same unit) defined, the result will be a fully-reduced shape holding the definition's UID. However, if the identifier is identified in another compilation unit, the result is a partially-reduced shape, because we cannot load the .cmt files of other compilation units (that are required to finish the reduction) without breaking the separate compilation assumptions. These resulting UIDs or partially-reduced shapes are stored in the unit's .cmt file:

type cmt_infos = {
  ...
  cmt_ident_occurrences :
    (ident_with_loc * def_uid_or_shape) list
}

Then, an external tool (called ocaml-index) will read that list and finish the reduction of the shapes when necessary. This step might load the .cmt files of any transitive dependency the unit relies on.

Indexation by the Build System

The tool we just introduced, ocaml-index, plays two roles:

1. It finishes the reduction of the shapes stored in the .cmt files.
2. It aggregates locations that share the same definition's UID.

The result is an index that is written to a file. Additionally, the tool can merge multiple indexes together. This allows build systems to handle indexation in the way they decide.

We only provide rules for Dune right now, but the tools themselves are built system agnostic. The Dune rules are as follow:

For every stanza library or executable, we index every .cmt file and store the results into an index file for the stanza.

• This process, similar to linking, depends on every transitive dependency of the stanza being indexed, since shape reduction might require loading those cmt files.
• Additionally, if any of the dependencies' indexes have changed, each stanza's index must be rebuilt.

This is a somewhat simple but heavy process, and it could be refined in the future. Right now it performs well enough to provide a usable watch mode in small to fairly large projects (like Dune itself).

Index Configuration and Reading

Last but not least, we need a way for Merlin to know were the index files are located and how to read them.

This is done by using a new configuration directive INDEX. It can be used to provide one or more index files to Merlin. Then, querying for all the usages of the identifier under the cursor is done in the following way:

• Identify the path of the identifier under the cursor
• Reduce the shape corresponding to this path to get the definition's UID.
• Lookup this UID in the index files and in the current buffer's index (which is computed by Merlin).
• Return all the locations

Future Work

Thank you for reading this post! We hope you will have a lot of fun exploring your codebases using this new feature. We have a lot of exciting improvements on our roadmap, some of which involve returning the declarations linked to an identifier and providing project-wide renaming queries.

If you are interested to learn more about these features or to contribute, please have a look at this tracking issue. You can also have a look at the announcement and wiki page. Finally, feel free to attend future Merlin public meetings and watch the talk at the OCaml Workshop during ICFP!

However, introducing such a significant change to the OCaml ecosystem would not be practical without providing tools that help users ensure memory safety in parallel programming. This is where multicore tests come in, as does ThreadSanitizer (TSan) support for OCaml. We have published two previous posts on TSan, an overview of the tool and an introduction to the danger of data races in general. This post will give you a behind-the-scenes look at how we have used TSan to find and fix races in the OCaml runtime. Official support for TSan arrived with OCaml 5.2, but as you can tell from the repository, we have been using TSan internally for a while now.

Catching Races in OCaml

As a result of TSan coming with the OCaml 5.2 update, several bug fixes in the same update addressed data races. A data race occurs when two accesses are made to the same memory location; at least one is a write, and no order is enforced between them. Data race bugs can be hard to spot, but since they can result in unexpected behaviours, they are a high-priority item to fix.

It is important to note that not all data races are equal. In OCaml, the memory model guarantees that memory safety is preserved even when data races occur. This makes data races in OCaml much 'safer' than in many other languages, where races can impact memory safety. OCaml programs also require support from the OCaml runtime which provides low-level operations such as memory allocation and garbage collection. The OCaml runtime is written in C and must be data race-free according to the C memory model since a data race in the runtime would impact the validity of the whole program. While TSan has been integrated to detect data races in OCaml code, it has also proven invaluable in detecting errors in OCaml's runtime, many of which were subsequently fixed in 5.2.

What has TSan Caught so Far?

Ecosystem contributors make continuous efforts towards maintenance, and as part of this work, use TSan to check the OCaml runtime. To further simplify TSan usage we have added a TSan Continuous Integration (CI) run that executes the OCaml test suite in a TSan-enabled switch, automatically detecting inadvertently introduced data races in the runtime. This has allowed us to catch and fix several data races. Some of these include:

• Fixing a Race in the Minor GC: PRs #12595 describes a race condition occurring when caml_collect_gc_stats_sample made calls to domain_terminate, and #12597 outlines the fix implemented in 5.2. In this data race, the internal garbage collector data could cause the program to report incorrect garbage collection statistics.

• Data Race Between Marking and Sweeping Garbage Collector Phases: PR #12934 fixes a race between marking and sweeping functions caught by TSan. When the garbage collector marks a value for collection, sweeping code (a later phase of the GC, effectively marking unreachable values as free space) in another thread may read the value simultaneously. This is normal behaviour, but the memory read was not marked as an atomic operation as it should have been, introducing a risk of undefined behaviour.

• Data Race on Global Pools Arrays: PR #12755 addresses races on global_avail_pools and global_full_pools members of the struct pool_freelist in shared_heap.c.
  For performance reasons, OCaml's runtime allocates major heap memory in chunks stored in pools that can be recycled across domains. While the algorithm for accessing them in parallel was correct, atomic qualifiers and explicitly qualified memory operations were missing, causing TSan to report unsynchronised memory accesses. The PR adds the necessary qualifiers.

• Data Races in minor_gc.c: This PR #12737 fixes two races. One in the minor GC occurring when promoting values in the remembered set, and one in the Dynlink library happening due to an incorrect C function trying to access an OCaml value even when the garbage collector might be running.

• Data Race fix for #12799: PR #12851 fixes a bug described in issue #12799. When a domain terminates, it emits a runtime event – a piece of information that can be monitored using a dedicated API to debug or profile the performance of OCaml programs. However, under certain circumstances, the emission of this event could race with the shutdown of the runtime events system itself, possibly leading to incorrect information being emitted or, worse, memory corruption. Proper synchronisation has been implemented to ensure this doesn't happen.

• Data Race When Using the Debug Runtime: PR #12969 resolves a data race involving caml_scan_stack and caml_free_stack. It was possible for two domains to perform a garbage collector marking on the same fibre, and in very rare cases when that fibre was terminating. This caused TSan to report a data race. While the code was correct in practice, we fixed the access to make it correct according to the C11 memory model, thus avoiding undefined behaviour.

Until Next Time!

Seeing a tool we have developed so quickly benefit the larger ecosystem is excellent. TSan helps developers test their parallel programs for potential risks they would have difficulty discovering. We look forward to seeing users put TSan to the test and share the results!

The OCaml community welcomes contributions and feedback and invites users to share any issues in the OCaml GitHub repo. The discussion forum OCaml Discuss is another place to share your thoughts and get input from others in the community.

Would you like to stay up-to-date with us? Follow us on Bluesky and LinkedIn to see regular posts on our projects, announcements, tutorials, and more.

While it is helpful to understand what happened with CrowdStrike, the next major outage will likely arise from a different flaw altogether. As such, and since we expect that another major event is likely to come, it’s essential to consider the risk factors behind these major cyber events and ways to reduce those risks.

The cybersecurity sector is constantly facing new threats and challenges. How can we transform these obstacles into opportunities for growth and improvement, ensuring greater protection for those who rely on our services? This blog post explores the answers to this question, presenting some lesser-known solutions that deserve consideration, and providing a fresh perspective on staying ahead of emerging threats and building more resilient defences.

What we Know so far

Affecting at least 8.5 million Windows machines, the outage significantly affected the aviation, broadcasting, and healthcare industry. The BBC labelled it “probably the largest ever cyber-event” and “one of the worst cyber-incidents in history.” But what actually happened?

Arising from something as commonplace as a software update for the Falcon platform, CrowdStrike’s own preliminary Post Incident Review, indicates that a security content configuration update delivered an undetected error (the now infamous Channel File 291) to user machines. The error slipped through the validation checks due to a bug, and trust in the tests allowed a faulty file with an out-of-bounds memory error to reach production. At its root, the global outage appears to be caused by unsafe parsers (a classic error), resulting in a parsing bug.

What are the Best Practices for the Cybersecurity Sector?

Now that we’ve covered this background, it’s time to look at some of the underlying factors that played a role in the CrowdStrike outage and the ones yet to come. The global outage served as an excellent wake-up call to the entire industry about the necessity of maintaining the checks and balances that keep our global systems secure.

Supply Chain Delivery

Let’s start with the fundamentals: how did the flawed update get delivered to so many? When deploying updates at scale, industry best practices can help mitigate risks by employing methods like staggered deployments and rollback options. In a best-case scenario updates are staggered, or rolled out incrementally, starting with 1% and, if all goes well, 5%, and so on. One can also use automated systems that rollback updates when a fault is detected, undoing the damage and keeping user’s machines operational. By using either method, preferably both in combination, a flaw in an update has negligible impact, voiding the kind of international chaos we saw on Friday.

Another critical aspect of the supply chain is the internal testing performed before deployment to ensure quality and safety. The recent chaos would likely have been preventable if the internal testing processes had caught the flaw earlier.While cost-cutting measures may be tempting, they can ultimately lead to much greater costs in the event of a cybersecurity breach or production downtime.

OS Monocultures: Do you Really Need a Full Windows OS Stack to run an Airport Screen?

The IT industry is increasingly dominated by a few major operating systems, primarily Microsoft Windows. This creates the emergence of 'OS monocultures', with one dominant provider overshadowing a few large ones (Linux and Mac), leaving little space for diverse, smaller suppliers. While these monocultures offer benefits, they also pose significant risks, and any introduced vulnerabilities can cause widespread damage.

This is what happened with CrowdStrike. The cybersecurity firm has around 20% market share, and because everyone uses the same stack, a single bug can have an enormous impact. One way to reduce the risk of a shared stack is to generate a unique stack for each application; that way, bugs are contained in that stack. One way of achieving this is by using unikernels to build a small, highly specified stack with only what is required to run the application. MirageOS is a library operating system that constructs unikernels to create secure, high-performance network applications with small attack surfaces.You don’t actually need to install a generic operating system to manage a single-purpose appliance, such as an airport screen.

Formal Verification, Testing, and Organisational Change

The road to security and reliability is paved with organisational changes that improve the overall stability of systems and reduce risk, impacting the way we develop software from start to finish. Scott Hanselmann, the VP of Developer Community at Microsoft, highlights this dynamic in one of his posts on X:

“It’s always one line of code, but it’s NEVER one person... Engineering practices failed to find a bug multiple times, regardless of the seniority of the human who checked that code in. Solving the larger system thinking SDLC matters more than the null pointer check.”

But what does changing the software development life cycle on an organisational level look like? For one, it involves spending the time and cost of creating comprehensive tests that catch bugs easily missed by developers before they ever reach production.

Including formal verification, for example of the device driver and the code it executes, is another aspect of software development that can prevent faults reaching production. Using formal verification, developers can mathematically prove that a program behaves according to its formal model and correctly performs a defined property.

As it relates to the CrowdStrike incident, formal verification of parsers is challenging but not impossible. For example, the EverParse framework emits secure, formally verified code for parsers that can be used in programs, including OCaml programs. Creating a software development culture that includes formal verification, fuzz testing, and other tests decreases the risk of failures slipping through the net.

The Role of Type Safety

Finally, let’s look at perhaps a more obvious topic in the light of the fault in Channel File 291. Using type- and memory-safe languages, like OCaml and Rust, prevents out-of-bounds memory errors and a whole class of other bugs.

That’s not the key takeaway we can all learn from, however, which is about complexity. The assertion that “the central enemy of reliability is complexity ... complex systems tend to not be entirely understood by anyone” from a cybersecurity paper authored by several industry specialists holds true in this case. By eliminating a whole class of errors at compile time, a language like OCaml with a strong type system critically reduces the kind of complexity that leads to cyber-insecurity. A language like OCaml simplifies the developer workflow when it comes to catching bugs.

Most importantly, from an industry standpoint, and as mentioned above, the faulty file that caused the outage remained undetected , even in the face of extensive stress tests. Building critical systems with secure-by-design principles includes using building blocks that contribute to the robustness of the entire system by preventing faults and reducing complexity. One facet of this puzzle is to use languages immune to certain kinds of bugs, such as type- and memory-safe languages, but that is not enough. Varied and rigorous tests, like fuzz testing, are a necessary complement to any language. For example, the MirageOS network stack has had fuzz testing performed on it to prevent parser issues, providing another layer of safety to the already type-safe OCaml.

Join in the Conversation

In essence, the cause and lessons from the CrowStrike outage are more complex than they may seem at first glance. It’s easy to reduce it to a line of faulty code, but the real takeaway is that the entire industry needs to implement better practices to safeguard users from risk. This outage was not the result of a cyber attack or malware, but the next one could be, and we cannot let the fate of our global networks rest entirely on endpoint security measures like antivirus programs, firewall management, and VPNs. We need to build foundational secure systems from the ground up.

In this context, and in light of global calls for change in how cybersecurity is addressed, it is the right time to have these conversations and strengthen the sector from within. We believe the best approach is to adopt a secure-by-design strategy implemented in a type-safe and reliable language like OCaml.

There are many perspectives on this, however, and we want to hear yours. Connect with us on Bluesky and LinkedIn and share your thoughts – we look forward to hearing from you!

In the first part of this series, Creating the SyntaxDocumentation Command - Part 1: Merlin, we explored how to create a new command in Merlin, particularly the SyntaxDocumentation command. In the second part, Creating the SyntaxDocumentation Command - Part 2: OCaml LSP, we looked at how to implement this feature in OCaml LSP in order to enable visual editors to trigger the command with actions such as hovering. In this third and final installment, you will learn how SyntaxDocumentation integrates into the OCaml VSCode Platform extension as an opt-in feature, enabling users to toggle it on/off depending on their preference.

VSCode Editor

Visual Studio Code is a free open-source, cross-platform code editor from Microsoft that is very popular among developers. Some of its features include:

• Built-in Git support
• Easy debugging of code right from the editor with an interactive console
• Built-in extension manager with lots of available extensions to download
• Supports a huge number of programming languages, including syntax highlighting
• Integrated terminal and many more features

OCaml Platform Extension for VSCode

The VSCode OCaml Platform extension enhances the development experience for OCaml programmers. It is itself written in the OCaml programming language using bindings to the VSCode API and then compiled into Javascript with js_of_ocaml. It provides language support features such as syntax-highlighting, go-to-definition, auto-completion, and type-on-hover. These key functionalities are powered by the OCaml Language Server (ocamllsp), which can be installed using popular package managers like opam and esy. Users can easily configure the extension to work with different sandbox environments, ensuring a tailored setup for various project needs. Additionally, the extension includes comprehensive settings and command options, making it very versatile for both beginner and advanced OCaml developers.

The OCaml Platform Extension for VSCode gives us a nice UI for interacting with OCaml-LSP. We can configure settings for the server as well as interact with switches, browse the AST, and many more features. Our main focus is on adding a checkbox that allows users to activate or deactivate SyntaxDocumentation in OCaml LSP's hover response. I limited this article's scope to just the files relevant in implementing this, while giving a brief tour of how the extension is built.

The Implementation

Extension Manifest

Every VSCode extension has a manifest file, package.json, at the root of the extension directory. The package.json contains a mix of Node.js fields, such as scripts and devDependencies, and VS Code specific fields, like publisher, activationEvents, and contributes. Our manifest file contains general information such as:

• Name: OCaml Platform
• Description: Official OCaml language extension for VSCode
• Version: 1.14.2
• Publisher: OCaml Labs
• Categories: Programming Languages, Debuggers

We also have commands that act as action events for our extension. These commands are used to perform a wide range of things, like navigating the AST, upgrading packages, deleting a switch, etc. An example of a command to open the AST explorer is written as:

{
    "command": "ocaml.open-ast-explorer-to-the-side",
    "category": "OCaml",
    "title": "Open AST explorer"
}

For our case, enabling/disabling SyntaxDocumentation is a configuration setting for our language server, so we indicate this in the configurations section:

"ocaml.server.syntaxDocumentation": {
    "type": "boolean",
    "default": false,
    "markdownDescription": "Enable/Disable syntax documentation"
}

Extension Instance

The file extension_instance.ml handles the setup and configuration of various components of the OCaml VSCode extension and ensures that features like the language server and documentation are properly initialised. Its key functionalities are:

• Managing the Extension State: It uses a record type that encapsulates the state of the extension, holding information about the sandbox, REPL, OCaml version, LSP client, documentation server, and various other settings.

type t = {
  mutable sandbox : Sandbox.t;
  mutable repl : Terminal_sandbox.t option;
  mutable ocaml_version : Ocaml_version.t option;
  mutable lsp_client : (LanguageClient.t * Ocaml_lsp.t) option;
  mutable documentation_server : Documentation_server.t option;
  documentation_server_info : StatusBarItem.t;
  sandbox_info : StatusBarItem.t;
  ast_editor_state : Ast_editor_state.t;
  mutable codelens : bool option;
  mutable extended_hover : bool option;
  mutable dune_diagnostics : bool option;
  mutable syntax_documentation : bool option;
}

• Interacting With the Language Server: This extension needs to interact with the OCaml language server (ocamllsp) to provide features like code completion, diagnostics, and other language-specific functionalities.

• Documentation Server Management: The file includes functionality to start, stop, and manage the documentation server, which provides documentation lookup for installed OCaml packages.

• Handling Configuration: This extension allows users to configure settings such as code lens, extended hover, diagnostics, and syntax documentation. These settings are sent to the language server to adjust its behaviour accordingly. For SyntaxDocumentation, whenever the user toggles the checkbox, the server should set the correct configuration parameters. This is done mainly using two functions set_configuration and send_configuration.

...

(* Set configuration *)
let set_configuration t ~syntax_documentation =
  t.syntax_documentation <- syntax_documentation;
  match t.lsp_client with
  | None -> ()
  | Some (client, (_ : Ocaml_lsp.t)) ->
      send_configuration ~syntax_documentation client
...

...

(* Send configuration *)
let send_configuration ~syntax_documentation client =
  let syntaxDocumentation =
    Option.map syntax_documentation ~f:(fun enable ->
        Ocaml_lsp.OcamllspSettingEnable.create ~enable)
  in
  let settings =
    Ocaml_lsp.OcamllspSettings.create
      ~syntaxDocumentation
  in
  let payload =
    let settings =
      LanguageClient.DidChangeConfiguration.create
        ~settings:(Ocaml_lsp.OcamllspSettings.t_to_js settings)
        ()
    in
    LanguageClient.DidChangeConfiguration.t_to_js settings
  in
  LanguageClient.sendNotification
    client
    "workspace/didChangeConfiguration"
    payload

...

Interacting With OCaml LSP:

The ocaml_lsp.ml file ensures that ocamllsp is set up correctly and up to date. For SyntaxDocumentation, two important modules used from this file are: OcamllspSettingEnable and OcamllspSettings.

OcamllspSettingEnable defines an interface for enabling/disabling specific settings in ocamllsp.

...

module OcamllspSettingEnable = struct
  include Interface.Make ()
  include
    [%js:
    val enable : t -> bool or_undefined [@@js.get]
    val create : enable:bool -> t [@@js.builder]]
end

...

The annotation [@@js.get] is a PPX used to bind OCaml functions to JavaScript property accessors. This allows OCaml code to interact seamlessly with JavaScript objects, accessing properties directly as if they were native OCaml fields, while [@@js.builder] facilitates the creation of JavaScript objects from OCaml functions. They both come from the LexFi/gen_js_api library.

OcamllspSettings aggregrates multiple OcamllspSettingEnable settings into a comprehensive settings interface for ocamllsp.

...
module OcamllspSettings = struct
  include Interface.Make ()
  include
    [%js:
    val syntaxDocumentation : t ->
      OcamllspSettingEnable.t or_undefined [@@js.get]

    val create : ?syntaxDocumentation:OcamllspSettingEnable.t ->
      unit -> t [@@js.builder]]

  let create ~syntaxDocumentation = create ?syntaxDocumentation ()
end
...

Workspace Configuration

The file settings.ml provides a flexible way to manage workspace-specific settings, including:

• Creating settings with JSON serialisation and deserialisation
• Retrieving and updating settings from the workspace configuration
• Resolving and substituting workspace variables within settings
• Defining specific settings for the OCaml language server, such as extra environment variables, server arguments, and features like codelens and SyntaxDocumentation

...
let create_setting ~scope ~key ~of_json ~to_json =
  { scope; key; to_json; of_json }

let server_syntaxDocumentation_setting =
  create_setting
    ~scope:ConfigurationTarget.Workspace
    ~key:"ocaml.server.syntaxDocumentation"
    ~of_json:Jsonoo.Decode.bool
    ~to_json:Jsonoo.Encode.bool
...

Activating the Extension

The vscode_ocaml_platform.ml file initialises and activates the OCaml Platform extension for VSCode. The key tasks include:

• Suggesting users select a sandbox environment
• Notifying the extension instance of configuration changes
• Registering various components and features of the extension
• Setting up the sandbox environment and starting the OCaml language server

In the context of SyntaxDocumentation, this code ensures that the extension is correctly configured to handle SyntaxDocumentation settings. The notify_configuration_changes function listens for changes to the server_syntaxDocumentation_setting and updates the extension instance accordingly. This means that any changes the user makes to the SyntaxDocumentation settings in the VSCode workspace configuration will be reflected in the extension's behaviour, ensuring that SyntaxDocumentation is enabled or disabled as per the user's preference.

let notify_configuration_changes instance =
  Workspace.onDidChangeConfiguration
    ~listener:(fun _event ->
      let syntax_documentation =
        Settings.(get server_syntaxDocumentation_setting)
      in
      Extension_instance.set_configuration instance ~syntax_documentation)
    ()

Conclusion

In this final article, we explored how to integrate SyntaxDocumentation into OCaml VSCode Platform extension as a configurable option for OCaml LSP's hover command. We covered key components such as configuring the extension manifest, managing the extension state, interacting with the OCaml language server, and handling workspace configurations. By enabling users to toggle the SyntaxDocumentation feature on or off, we can ensure a flexible and customisable development experience for all users.

Feel free to contribute to this extension on the GitHub repository: vscode-ocaml-platform. Thank you for following along in this series, and happy coding with OCaml and VSCode!

Challenge

The OCaml HTML manuals have URL references such as https://v2.ocaml.org/manual/types.html#sss:typexpr-sharp-types, and they do not refer to any specific compiler version. We needed a way to easily share a link with the version number included. The OCaml.org page already has a mention of the compiler version, but it refers to specific https://ocaml.org/releases.

We wanted a canonical naming convention that is consistent with current and future manual releases. It would also be beneficial to have only one place to store all the manuals, and the users of OCaml.org should never see redirecting URLs in the browser. This will greatly help increase the overall backlink quality when people share the links in conversations, tutorials, blogs, and on the Web. A preferred naming scheme should be something like:

https://v2.ocaml.org/releases/latest/manual/attributes.html https://v2.ocaml.org/releases/4.12/manual/attributes.html

Using this, we redirected the v2.ocaml.org to OCaml.org for the production deployment. Also, the changes help in shorter URLs that can be easily remembered and shared. The rel="canonical" is a perfectly good way to make sure only https://ocaml.org/manual/latest gets indexed.

Implementation

After a detailed discussion, the following UI mockup to switch manuals was provided via GitHub issue, and Option A was selected.

Our proposed changes to the URL are shown below:

Current: https://v2.ocaml.org/releases/5.1/htmlman/index.html
Suggested: https://ocaml.org/manual/5.3.0/index.html

Current: https://v2.ocaml.org/releases/5.1/api/Atomic.html
Suggested: https://ocaml.org/manual/5.3.0/api/Atomic.html

HTML Compiler Manuals

The HTML manual files are hosted in a separate GitHub repository at https://github.com/ocaml-web/html-compiler-manuals/. It contains a folder for each compiler version, and it also has the manual HTML files.

A script to automate the process of generating the HTML manuals is also available at https://github.com/ocaml-web/html-compiler-manuals/blob/main/scripts/build-compiler-html-manuals.sh. The script defines two variables, DIR and OCAML_VERSION, where you can specify the location to build the manual and the compiler version to use. It then clones the ocaml/ocaml repository, switches to the specific compiler branch, builds the compiler, and then generates the manuals. The actual commands are listed below for reference:

echo "Clone ocaml repository ..."
git clone git@github.com:ocaml/ocaml.git

# Switch to ocaml branch
echo "Checkout $OCAML_VERSION branch in ocaml ..."
cd ocaml
git checkout $OCAML_VERSION

# Remove any stale files
echo "Running make clean"
make clean
git clean -f -x

# Configure and build
echo "Running configure and make ..."
./configure
make

# Build web
echo "Generating manuals ..."
cd manual
make web

As per the new API requirements, the manual/src/html_processing/Makefile variables are updated as follows:

WEBDIRMAN = $(WEDBIR)/$(VERSION)
WEBDIRAPI = $(WEBDIRMAN)/API

Accordingly, we have also updated the manual/src/html_processing/src/common.ml.in file OCaml variables to reflect the required changes:

let web_dir = Filename.parent_dir_name // "webman" // ocaml_version

let docs_maindir = web_dir

let api_page_url = "api"

let manual_page_url = ".."

We also include the https://plausible.ci.dev/js/script.js script to collect view metrics for the HTML pages. The manuals from 3.12 through 5.2 are now available in the https://github.com/ocaml-web/html-compiler-manuals/tree/main GitHub repository.

OCaml.org

The OCaml.org Dockerfile has a step included to clone the HTML manuals and perform an automated production deployment as shown below:

RUN git clone https://github.com/ocaml-web/html-compiler-manuals /manual

ENV OCAMLORG_MANUAL_PATH /manual

The path to the new GitHub repository has been updated in the configuration file, along with the explicit URL paths to the respective manuals. The v2 URLs from the data/releases/*.md file have been replaced without the v2 URLs, and the manual /releases/ redirects have been removed from redirection.ml. The /releases/ redirects are now handled in middleware.ml. The caddy configuration to allow the redirection of v2.ocaml.org can be implemented as follows:

v2.ocaml.org {
	redir https://ocaml.org{uri} permanent
}

Call to Action

You are encouraged to checkout the latest OCaml compiler from trunk and use the build-compiler-html-manual.sh script to generate the HTML documentation.

Please do report any errors or issues that you face at the following GitHub repository: https://github.com/ocaml-web/html-compiler-manuals/issues

If you are interested in working on OCaml.org, please message us on the OCaml Discord server or reach out to the contributors in GitHub.

References

1. (cross-ref) Online OCaml Manual: there should be an easy way to get a fixed-version URL. https://github.com/ocaml/ocaml.org/issues/534

2. Use webman/*.html and webman/api for OCaml.org manuals. https://github.com/ocaml/ocaml/pull/12976

3. Serve OCaml Compiler Manuals. https://github.com/ocaml/ocaml.org/pull/2150

4. Simplify and extend /releases/ redirects from legacy v2.ocaml.org URLs. https://github.com/ocaml/ocaml.org/pull/2448

Sometimes, we also find ourselves just trying to keep up with our interns as they take off like rocket ships! Recently, we mentored a student who did just that. The initial goal of the internship was to investigate strange performance drops in the OCaml runtime that arose after the introduction of multicore support. These performance drops were most keenly felt on Windows machines, and the initial internship specification emphasised the need to improve the developer experience on that operating system.

Our intern @eutro went above and beyond anything we could have expected and tackled the project thoroughly and ambitiously. In this post, I will attempt to give you a comprehensive overview of this intricate project and the problems it tackled.

Get Busy Waiting?

Before OCaml 5, only one thread would run at any given time. Users never had to worry about multiple threads trying to use a shared resource like the Garbage Collector (GC). In OCaml 5, however, the process is divided into several 'threads'^[1], and multiple threads regularly try to run parts of the GC simultaneously. The minor GC uses a Stop The World (STW) function to run in parallel on all threads, whereas the major GC’s work is split into slices. These may happen in parallel between threads and while the user’s program (also called the ‘mutator’) is making changes. This is one example of when a mechanism is needed to protect multiple threads from making changes that contradict each other and result in unexpected behaviours.

Locks are the traditional way of doing this, whereby other activity is halted (or locked) while one activity finishes. However, in multicore programming, this method would be incredibly inefficient since there can be many activities in progress simultaneously. In this case, we would need to introduce so many locks for the different parts of memory that doing so would cause memory and OS resource problems!

The approach we use for OCaml 5 combines a Compare And Swap (CAS) operation with Busy-Wait loops. A CAS operation ensures that if two threads try to modify the same area of memory, only one will succeed. The one that fails will know it has failed and can then enter a period of Busy-Waiting (called SPIN_WAIT in the code). Busy-wait loops (also referred to as spins) describe a process that repeatedly ('busily') checks whether a condition is true. The process or task is only resumed once that condition is met.

Sleeping Beauty

Busy-wait loops are used successfully in OCaml for many purposes but have been optimised. They are mostly appropriate in cases where we think that the required condition will be met quickly or in a reasonable period of time. If that’s not the case, then theoretically, the thread that is waiting will just keep spinning. If one allows busy-wait loops to spin indefinitely, they waste a lot of power and CPU and can actually prevent the condition they are waiting for from being met. To avoid that happening, we can use a sleep function.

In order to implement spinning without wasting power, the loop checks the condition repeatedly, but after a while, it starts 'sleeping' between checks. Suppose a thread is waiting for condition C to come true, and it uses a Busy-Wait loop to check for this. The program spins a number of times, checking the condition, and then waits or goes to ‘sleep’ for a set amount of time – then it ‘wakes up’ and checks once more before (if it has to) going back to ‘sleep‘ again. The period of ‘sleep’ increases each time. This cycle repeats itself until the condition C finally comes true.

This was how the process was supposed to work, yet, for some unknown reason, certain processes would occasionally take much longer than expected. The performance drop was worst on Windows machines.

Testing 1-2-3, Testing 1-2-3,

The first order of business was to conduct a series of tests on the runtime. Not only to discover the possible cause of the performance drops but also to establish a baseline of performance against which to measure any changes (and hopefully improvements!).

We knew that there was a performance problem and that it was particularly painful on Windows, but we didn’t know why. Even if we had a hunch as to what might be causing it, it was crucial to build up a realistic image of what was happening before we attempted a fix.

@eutro began this process by identifying where Busy-Wait loops were spinning in the runtime and for how long. She also wanted to know if there were places in the runtime where processes would get ‘stuck’ in Busy-Wait loops and not move on, and if so, where and why.

She used the OCaml testsuite and measured how many SPIN_WAIT macros resolved successfully without needing to sleep and which ones did not. She discovered that in most cases, the spinning had the desired effect, and the process could continue after a reasonable amount of time when the condition it was waiting for was met. The garbage collector was also not experiencing any significant performance drops, so it could not be the cause of the problems on Windows. Instead, what she realised was that on Windows, sleeps cannot be shorter than one millisecond, and so the first sleeps that occur end up being much too long. This causes extended and unnecessary delays for processes running on Windows. Equipped with this realisation, @eutro got started on a solution. One that would be most helpful on Windows but still benefit users on other operating systems.

Barriers and Futexes, Oh My!

There are a few ways a thread in OCaml can wait for a condition:

• First, we may be able to proceed very soon (nanoseconds), in which case we will spin until we can proceed.
• Then, if spinning doesn’t let us proceed, we sleep a few times until the condition comes true. In most cases (read: not on Windows), the first few sleeps are still very quick, and we can proceed soon once the condition is met.
• An alternative to sleeping ‘blindly’ like this is to tell the OS specifically what we are waiting for so that we can be woken up only when we know the condition is true. You can think of this as taking a ticket and waiting for your number to be called rather than repeatedly asking if you can be seen.

So what has changed? As things stood, only steps one and two were available, a series of increasingly long sleeps interleaved with checks. So you would spin n times, then sleep for 10µs (‘µs’ is short for microseconds), then you check the condition once more and might sleep for 20µs, then 35µs, and so on. The point is that the time spent sleeping kept gradually increasing.

However, as @eutro discovered, in many cases, the process took far too long to resume, even after the condition had come true. By the time they woke up from sleeping, they could have already proceeded if they had just ‘taken a ticket’ earlier and waited until they were notified. To improve performance, instead of repeatedly sleeping for longer increments, we use specialised ‘barriers’ to wait until we can proceed.

To solve the Windows problem, we now use the SPIN_WAIT function only in cases where we don’t expect to ever need to sleep. In cases where that first sleep would cause significant delay, we introduce a new SPIN_WAIT_NTIMES function, which lets the process spin for a set number of times before being combined with a barrier. @eutro used her previous benchmarks to determine which occasions could keep the SPIN_WAIT cycle as-is and which occasions required the new SPIN_WAIT_NTIMES combined with a barrier.

But things didn’t stop there! @eutro could also optimise the type of barrier. Traditionally, we use condition variables to wake up threads waiting on a condition. However, they are unnecessarily resource-intensive as they require extra memory, and since woken threads must acquire (and release) a lock before they continue. A futex is a lower-level synchronisation primitive that can similarly be used to wake up threads but without the added complexity of a condition variable.

@eutro added the use of futexes to the operating systems that permitted her to do so. Notably, macOS does not allow non-OS programs to use futexes so we fall back to using "condition variables" there.

By introducing the use of SPIN_WAIT_NTIMES, barriers, and futexes, @eutro implemented a number of optimisations that were applicable not only on Windows but on several operating systems. These optimisations save users time and processing power.

How Much do You Bench(mark)?

During the course of implementing these changes, @eutro did a lot of tests. It was important to be thorough in order to ensure that her changes didn’t have unintended consequences. It is incredibly difficult to reason about how programs will react to a specific alteration, as there are many things happening in the runtime and several ways that programs can interact.

She used the OCaml test suite again, this time to help her check that the OCaml runtime and other OCaml programs functioned correctly. Having verified that they were, @eutro also ran several benchmarks to check that she hadn’t actually made anything slower. For this, she used the Sandmark test suite.

I recommend checking out the tests and benchmarks for yourself in the Pull Request. The PR also gives a more in-depth technical overview of the changes to the Busy-Waiting loops.

You Can Join Us Too!

It is great to see what someone with a passion for OCaml can bring to the system as a whole. I think it illustrates the benefits of open-source software development: when we invite fresh observations and suggestions, we create a community that supports innovation and collaboration. We are impressed with the hard work @eutro put into solving the complicated problem before her. She went above and beyond what we thought possible in such a short amount of time!

Would you like to complete an internship with us? We welcome people of varying experience levels – some interns have made open-source contributions before and are familiar with functional programming, and some of our interns have no functional programming experience at all! If you’re interested in developing your skills in a supportive environment, keep an eye on our careers page, where we post about any available internships. We also regularly communicate about available internships on Bluesky. We hope to hear from you!

We recently introduced you to the monitoring system runtime_events, which allows users to monitor their runtime for, among other things, how programs are affecting performance. Alongside runtime_events, sits the observability tool olly, which provides users with a number of helpful formatting options for their runtime tracing data.

This is all part of how we’re making developing in OCaml easier by bringing new features and tools to the community. Olly is just one such tool, and it makes the monitoring system for OCaml significantly more accessible. With Olly, you don’t have to be an expert or spend time combing through the data that runtime_events extracts for you. Rather, Olly can generate the information you need in a way that makes it easy to understand, store, and query.

What is Olly and How Does it Work?

Olly, as an observability tool for OCaml 5, has the ability to extract runtime tracing data from runtime_events. This data can then be visualised with a variety of graphical options available.

How does Olly do this? Olly uses the Runtime API to provide you with monitoring metric information and associated data. The tool comes with several subcommands, each with its own function.

The command olly trace can generate runtime traces for programs compiled in OCaml 5 using its trace subcommand. The tracing data is generated in one of two formats, the Fuschia trace format or the Chrome tracing format with the former being the default. Both formats can be viewed in Perfetto, but the Chrome format trace can also be viewed in chrome://tracing for Chromium-based browsers. Another example of a subcommand is olly gc-stats, which can report the running time of the garbage collector (GC) and the GC tail latency of an OCaml executable.

The motivation behind introducing an observability tool like Olly is to make data extracted using runtime_events more useful, since few developers will want to use the event tracing system directly. Olly makes it easy for users to troubleshoot their own programs, but it also makes it easy for a developer to diagnose why someone else’s program is slow. A client can send their runtime_events data, a feature that comes built in with every OCaml 5 switch, to a developer who can then use Olly to find the problem and suggest a solution. This makes working in OCaml is easier as optimisation and problem solving becomes more efficient and streamlined.

It doesn’t end there! One of our future goals for Olly is that it should be able to provide automatic reports and diagnosis of some problems. Look out for that exciting update in the future!

Recent Update: Modularising Olly

One of the latest updates to Olly is its modularisation by Eutro, splitting the bin/olly.ml file into smaller discrete libraries including olly_common, olly_trace, and olly_gc_stats. By splitting up the large file, the user can exercise some control over which dependencies they want their library to have. They can create a minimal build with minimal dependencies, or stick with a fuller build relying on all the dependencies. For example, to build olly_bare on the trunk you now only require two dependencies: Dune and cmdliner. Both can be installed without using Opam. Since some developers will prefer this set up, it’s good to support a variety of configurations.

It also potentially makes it easier to maintain, since the smaller files have well-defined purposes and provide a clearer overview than just having one large file covering a multitude of functions. If something breaks, this segmentation can make it easier for a maintainer to triage and amend the problem. The same modularisation may also help newcomers get an overview of all the different components of the library. Sadiq Jaffer merged Eutro’s PR #43 into Tarides: main and it will form part of a future Olly release pending further testing.

How to Use Olly: an Example

Let's wrap up by looking at an example of when you might use Olly. When we want to visualise the performance of the OCaml runtime alongside any custom events we may have, the first step is to generate a trace. To generate a trace, we run the command olly trace tracefile.trace in combination with the name of the program we want to enable tracing for. If we wanted to generate a trace for the solver-service, the command would be olly trace tracefile.trace 'solver-service'.

For our example, we chose to generate the tracing data in the Fuschia trace format. Once we had the trace, we loaded it into Perfetto to get a helpful visual representation of what our code is doing and we ended up with the following image:

The UI in this image displays the processes down the side, each corresponding to a domain. Our program ended up using four cores, and therefore, the image shows four processes. Each process, in turn, shows the tracing for the OCaml runtime build plus the custom events generated by Eio. Let's zoom in on one process now:

This expanded view shows both the Garbage Collector's (GC) activity and times when Eio is suspended.

Until Next Time!

We want to create tools that make the developer experience in OCaml easier and more intuitive. Olly makes it possible to visualise your code's performance, helping you understand when your programs are slowing down and why. If you have suggestions or improvements to share, you are welcome to participate in the Runtime Events Tools repo on GitHub.

We want to hear from you! Connect with us on social media by following us on Bluesky and LinkedIn. You can also join in with the rest of the community on the forum Discuss to share your thoughts on everything OCaml!

The OCaml community covers various domains, helping the language grow and supporting developers in their careers. To better serve our users, we decided to redesign the community area's concept pages. We conducted surveys, online discussions, and video calls to gather feedback and insights for this project.

Our redesign targets key UX priorities. The navigation has been reorganised for easy content access, and the landing page now highlights essential community features. Additionally, we have revamped other pages, including events and job listings, to enhance engagement and activation within the OCaml community.

Navigation

Consistency is a fundamental aspect of our navigation design. As discussed in a previous post, we have implemented a main navigation bar that provides subnavigation options for user consistency. This approach allows users to easily access events, blogs, job resources, and more with a single click, avoiding the need to scroll through a lengthy landing page with too many options.

For experienced users, this streamlined navigation meets their clear objectives when visiting the community area. Beginners, on the other hand, can benefit from an overview of what OCaml.org offers with highlighted subsections that showcase real-life community activities. This intuitive navigation ensures all users, whether newcomers or advanced users, can find what they need efficiently and effectively.

By focussing on these improvements, we aim to create a more engaging and user-friendly community page that supports the growth and development of the OCaml community.

Landing Page

We rearranged the landing page to retain important key elements from the current design, such as the community channel. We reorganised the rest of the page and added new features to boost user engagement and provide valuable resources.

Key Updates

• Social Media Channels: The existing list of social media channels remains a focal point due to its critical role in user interaction and collaboration.

• Upcoming Events: We introduced a section for upcoming events, encouraging users to participate in retreats, conferences, and meetups. This addition aims to foster greater community involvement and real-world interaction.

• Job Opportunities and Resources: Featuring job listings, internships, and essential documents for newcomers helps users find career opportunities and resources to aid their professional development.

• Code of Conduct: We prominently displayed the community's code of conduct to remind users of the importance of respect and quality. This ensures a positive and collaborative environment, emphasising that the community is built by and for its members.

By implementing these changes, our goal is to create a more engaging, informative, and user-friendly landing page that supports the OCaml community's growth and development.

Inner Pages

Behind the community landing page, we've enriched the content with more detailed offerings to enhance user engagement and information accessibility.

The Events page now lists all upcoming events, which users can filter by location and event type. Additionally, it features recurring events that happen at least once a year in various locations, bringing the global community together and fostering real-life interactions and discussions.

The Job Board page is a valuable resource for newcomers, showcasing the types of companies that use OCaml. Job offers are displayed by location, making it easy for users to find relevant opportunities. Companies can also add their job offers to the board, promoting their roles and increasing visibility within the OCaml community.

The Outreachy Program section highlights real-life collaboration opportunities for newcomers, helping them discover and engage with the OCaml language. We maintain a record of past internships, offering resources and blog posts that can capture the interest of specific users. This provides valuable insights and inspiration for those looking to get involved.

The OCaml Conferences and Workshops pages showcase both upcoming and past conferences and workshops, providing a comprehensive record that includes valuable documentation such as videos and slides. Speakers can submit talks, and all sessions are recorded and made accessible anytime, which makes this page an excellent resource for deep diving into specific subjects. This extensive archive allows developers to learn from expert discussions and presentations to enhance their knowledge and engagement with the OCaml community.