Data Apps in F# or F# in Data Apps?

🎄 ♫ Thiiissss Yeeaaar, to save me from tears ♫ instead of showing how to do something cool in F#, I briefly show how to use F# elsewhere.

Observable Platform

Observable Platform introduced the Framework, an open-source static site generator, in February. In September, they announced Observable Cloud and recently introduced embedded analytics. I’ve played with it over the past few days, and I can admit that their slogan about effortlessly hosting data apps is true.

Since we can securely host one data app (or blog) for free, it would be exciting to use F# and Fable on a platform with 250k+ users, alongside one of the best libraries ever created: D3.js.

The Framework supports polyglot programming (to some extent) with many exciting features yet to come. Apparently, it may be one of the easiest and most gradual ways to learn F# and Fable available. It doesn’t enforce using F# for everything from scratch and avoids the hassle of SPAs.

I will show how to:

• use F# as a data loader for the static site generator
• create simple data visualization in F#
• use Fable for live previews along the Observable Framework
• describe what will be soon possible

I'm planning to extend this post as I have more time for further adventures, so feel free to revisit it from time to time.

Example visualization

I assume you already have a valid reason to use F# instead of, or side-by-side JavaScript, and this post is not about advocating for that choice.

D3.js is a low-level library that allows you to display anything you want, in any way you want. By using F#, you can gain an advantage by seamlessly mixing the domain logic you address with the low-level intricacies of visualization—not the bindings themselves, which are, in the vast majority of cases, a 1:1 translation from JavaScript.

For demonstration purposes, I’ve ported one of the D3JS zooming examples. However, your data app can be anything, even a blog. I’ve chosen this example purely to illustrate how to use F# within the Observable Framework:

import { smoothZoom } from './fs/Component.fs.js'
smoothZoom(count);

Note that although I'm referencing a .js file, the visualization is entirely generated from F# code transpiled to JavaScript. In future releases, possible the very next one, we should have the ability to use other languages directly in Markdown.

Why choose Data Apps ?

Starting our web programming adventures comes with a high cognitive overhead. JavaScript, routing, bundling, state management, React/SolidJS or any other framework, and so much more—building modern web applications is inherently complicated.

We might think that by replacing JavaScript with F#, a robust and succinct language, we can tackle this complexity by using the same tool for both coding the target domain and managing web internals.

While I believe this is true, we will not eliminate any of the required knowledge. Additionally, we need bindings, more development tools, and must spend a lot of time practicing.

Many people who know F# may give up at the very start when they encounter initial descriptive error messages.

The reason is that the vast majority of web apps is SPA (Single Page Applications) that are used for intensive data processing.

SPA's involves user actions to load content dynamically and maintain application state on the client side across different views. While many scenarios cannot be implemented otherwise, applications oriented toward displaying data can.

Observable Framework

Observable Framework is an open-source static-site generator for data apps.

You can:

• install it by npx "@observablehq/framework@latest" create
• run by npm run dev
• deploy by npm run deploy

Standard flow for any web application, but before explaining how F# and Fable can fit into the picture, let's briefly look at data loaders.

Data loaders

Although we can use plain fetch or websockets to get the data on load, we should first ask ourselves if a plain data snapshot is enough to avoid all that unnecessary overhead of overengineered SPA applications that could have been a static site.

Data loaders generate static snapshots of data during build and can be defined in any language.

To use F# we have to modify observable.config.js file:

{
    "interpreters": {
        ".fsx": ["dotnet fsi", "--quiet"],
        ...
    }
}

It does not have to be fsi but any executable that outputs data in a json/csv or other common format.

In my example I use an overengineered script that generates initial, random number of circles (just to show it can really be any F# code)

#r "nuget: FsHttp"

open FsHttp
Fsi.disableDebugLogs()

http {
    GET "https://www.randomnumberapi.com/api/v1.0/random?min=1&max=1000&count=1"
}
|> Request.send
|> Response.toJson

|> fun x -> printf $"{x[0]}"

Now, if you type the following in your Markdown:

    ```js
    const data = FileAttachment("fs/data/zoom.json").json();
    ```

the Framework will search for a file named zoom in any of the configured languages. In our case, it will find and execute zoom.json.fsx. This process happens during the build, and the result is cached. This makes the data app highly performant.

We can now take the generated data snapshot and pass it to our component written in F#:

    ```js 
    import { smoothZoom } from './fs/Component.fs.js'
    smoothZoom(data);
    ```

We can experiment with the code of the data loaders and still see the live preview:

Using Fable

Below is the ported D3.js example in F#. It looks almost identical to JS version. I didn't spend too much time on it hence you can still see the ? operator somewhere. Less important parts are hidden in modules, see the source code if you are interested.

module Project

open Utils
open D3JS

open Fable.Core.JsInterop

let smoothZoom (items: int) =
     
    let element = d3.select "#smooth"
    let g = element.select "g"
    let bounding = element?node()?getBoundingClientRect()
    let width = bounding?width 
    let height = bounding?height 

    let mutable currentTransform =
        [|
            width / 2.
            height / 2.
            height
        |]

    let theta = π * (3. - sqrt(5.))
    let radius = 6.
    let step = 2. * radius

    let data =
        [| 0 .. items |]
        |> Array.map (fun i ->
            let r = step * sqrt(i .+ 0.5)
            let a = theta .* i
            [|
                width / 2. + r * cos a
                height / 2. + r * sin a
            |]
        )

    let transform([| x ; y; h |]: float[]) =
        $"""
          translate({width / 2.}, {height / 2.})
          scale({height / h})
          translate({-x}, {-y})
        """
    g
     .selectAll("circle")
     .data(data)
     .join("circle")
        .attr("cx", fun (x: float[]) -> x[0])
        .attr("cy", fun (x: float[]) -> x[1])
        .attr("r", radius)
        .attr("fill", fun (_,i) -> d3.interpolateRainbow (i ./ 360.) ) |> ignore

    let rec transition() =
 
        let index = rand data.Length
        let [| x; y |] = data[index]

        let i = d3.interpolateZoom (currentTransform, [| x; y; radius * 2. + 1. |])

        g.transition()
         .delay(250<ms>)
         .duration(i.duration)
             .attrTween("transform", fun () -> fun t ->
                 currentTransform <- i.tick t
                 transform currentTransform
             )
         .on("end", transition)

    element.call transition

First and foremost, I assume that our primary focus for changes here is the Observable Framework, with the F# code being a secondary consideration.

If the framework, as a static site generator, needs to be updated, we want to avoid breaking it by embedding F# code too deeply into its structure.

I keep my F# code in a dedicated F# project, which I place in the src\fs directory of the Observable project, allowing me to reference the transpiled .js components in the markdown.

I also keep the Observable application as an attached folder next to the F# project, so everything is accessible in Rider, which displays the Markdown nicely.

These are just my initial experiments after a few days of exploration.

package.json

To support Fable we have to modify package.json of the Observable project:

{
    "type": "module",
    "private": true,
    "scripts": {
        "clean": "rimraf src/.observablehq/cache",
        "build": "observable build",
        "dev": "observable preview",
        "fable": "fable src/fs --watch", // <--- added fable watch
        "start": "concurrently \"npm run fable \" \"npm run dev\"", // <--- run fable along the Framework
        "deploy": "observable deploy",
        "observable": "observable"
    },
    "dependencies": {
        "@observablehq/framework": "^1.13.0",
        "concurrently": "^9.1.0", // <--- used in npm run start above
        "d3": "^7.9.0",
        "d3-dsv": "^3.0.1",
        "d3-time-format": "^4.1.0"
    },
    "devDependencies": {
        "rimraf": "^5.0.5"
    },
    "engines": {
    "node": ">=18"
    }
}

Now by running npm run start we will have both fable and observable working in a preview mode, if we modify anything, we will see changes instantly:

Inputs

In my example, I use the range input, and it works seamlessly with the F# data loader and F# components

const data = FileAttachment("fs/data/zoom.json").json();

const count = view(Inputs.range([1, 2000], {label: html`Number of <i>circles</i>`, step: 1, value: data}));

import { smoothZoom } from './fs/Component.fs.js'
smoothZoom(count);

What about deployment?

As long as we use isolated components or functions, we have to do nothing. We can just use our F# component as follow somewhere in the markdown

    ```js 
    import { smoothZoom } from './fs/Component.fs.js'
    smoothZoom(data);
    ```

and the framework will generate the dist. With npm run deploy you will have the application online.

Random thoughts

1. Read the documentation and explore what’s possible with both the Framework and the Cloud. The Framework supports reactivity, offers many input controls, routing, and libraries
2. Using React or other frameworks is also possible. In fact, this blog post was delayed because I wanted to include a React and Elmish example, but I encountered a bug that took several days to notice. Although the bug is now fixed, the update hasn’t been released yet. I also plan to experiment with Oxpecker and Solid JS
3. You can create unlimited Observable Notebooks and convert them into Markdown. Perhaps, at some point, we’ll even have the ability to use F# directly within those notebooks.
4. Checkout the embedded analytics. It’s currently available only for Enterprise users, but I think it’s a killer feature, allowing you to use your data apps almost natively on external hosts.
5. In my example, I use JavaScript code blocks that reference .fs.js files, but soon it will be possible to use custom tags transpiled behind the scenes. In our case, this will involve fable-compiler-js, and I’ll explore it as soon as it becomes available.
6. More thoughts to come ...

Summary

This blog post is a part of F# Advent Calendar in English 2024.

Many thanks to Sergey Tihon for organizing this event for so many years.

Happy Holidays, everyone!