The matter at hand boils down to a simple Haskell program which serves static files (with some extra steps regarding authorization) plus an nginx reverse proxy which proxies requests to a certain “virtual host” to the Haskell program.

1 First, Panic

Not even minutes after I announced that the files were finally available, hundreds of customers visited my server all at once. As the logs started flying off of my screen with all the accesses, I started noticing a particularly interesting message, repeated over and over again:

Mar 31 20:43:03 mogbit kanjideck-fulfillment[2528300]: user error
(Unexpected reply to: MAIL "<...> at kanjideck.com",
Expected reply code: 250, Got this instead: 452 "4.3.1 Insufficient system storage\r\n")

Oh no. No one’s able to access their files and I’m already receiving emails about it. I messaged the users explaining the server was having some issues that I was resolving.

Grafana shows 40GB/40GB disk space used up, so does df -h have 100% usage of /dev/sda. I have to clear up space fast. I’m afraid at this point that I’m not even receiving the user complaints anymore since my mail could be getting dropped by lack of space.

I rushed to run du -sh on everything I could, as that’s as good as I could manage. The two larger culprits were /var/lib’s Plausible Analytics, with a 8.5GB (clickhouse) database, and the /nix/store with the full server configuration, installation, and executables, at 15GB.

(In hindsight, I should have stopped right here to think carefully about what could possibly be occupying the remaining 20GB. I assumed “the rest of the files”, but looking back the “rest of the files” could hardly total 20GB.)

Delete everything that I can.

First off, the /nix/store may have unnecessary executables and past configurations. This should be a big win. Drop it all with

$ nix-collect-garbage -d
...
removing old generations of profile /nix/var/nix/profiles/system
error: opening lock file '/nix/var/nix/profiles/system.lock': No space left on device

Clearing the nix store doesn’t work because there’s no space left on device (meanwhile, imagine the panic as the errors scroll by as users try over and over again to download the files which keep giving them errors).

OK, let’s kill the logs first.

journalctl --vacuum-time=1s

This restored enough space for me to clear the nix store. Next big item on the list is the clickhouse database. Some googling tells me I can truncate some of the logs tables to reduce the size. Let’s try it:

clickhouse-client -q "TRUNCATE TABLE system.query_log"
  Received exception from server (version 24.3.7):
  Code: 243. DB::Exception: Received from localhost:9000. DB::Exception: Cannot reserve 1.00 MiB, not enough space. (NOT_ENOUGH_SPACE)
  (query: TRUNCATE TABLE system.query_log)

Aaaand we’ve run out of space again. The service is still returning errors left and right to the users. Maybe the nix store didn’t clear up enough space, but there’s clearly something that keeps consuming more and more space, correlated with the influx of users.

2 Mounting the Nix Store on a separate Volume

When in panic, buy more space. Except that Hetzner didn’t have an available cloud instance with more space for me to upgrade to.

Plan B: I could still buy more space as a separate Volume.

The /nix/store is an immutable store and I had heard of people setting up their nix stores on separate drives before. It was also the largest system component at 12GB now. A perfect candidate.

Luckily (rather, due to NixOS) everything went smoothly with this transition. Following the instructions on “Moving the store” in the NixOS Wiki worked flawlessly. The new Volume was labeled nix with mkfs.ext4 -L nix /dev/sdb and the mounting migration first done manually, but at the end of the day we have a final declarative configuration of the system:

  fileSystems."/nix" = {
     device = "/dev/disk/by-label/nix";
     fsType = "ext4";
     neededForBoot = true;
     options = [ "noatime" ];
   };

After rebooting the server, the /nix/store was living in a separate volume and the root drive finally had enough space to reply to the users.

Grafana was no longer red all over and the logs were no longer streaming error messages. The filesystem was still 50% used up and it seemed to increase up to around 60-65% when various users were downloading the large 2.2GB file. But. Working.

3 Finding the root cause in Nginx

This morning I anxiously opened my inbox. There were a handful of complaints about the large 2.2GB file download stopping halfway through and never successfully downloading.

However, users were able to access the download page and download all the other (arguably more important) files. Not bad! Grafana was still green and filesystem usage at about 50%.

The large file bug was important to fix promptly.

• Recall from the server architecture that nginx proxies to the program which serves the files.

• With some investigation I found proxy_max_temp_file_size, which defaults to

  Default: proxy_max_temp_file_size 1024m;

• Increasing the value to 5000m allowed the 2.2GB file to be served successfully by the proxy!

• Have you read the documentation for this option? I just skimmed.

One bug down. And the disk space issue seemed tamed, but… during the day, it briefly spiked up to 100% again! Without last night’s huge pressure I was able to investigate this more soundly.

The lsof +L1 command finds unlinked open files (see man lsof), i.e. files to which there are no links from the file system but are still referenced by some process and thus can’t be collected. Files which wouldn’t ever show up with ds -h. I was greeted by 14.5 GB of deleted files held by nginx!

[nix-shell:~]# lsof +L1 | grep nginx
  nginx     4659       nginx mem    REG    0,1   10485760     0    1187 /dev/zero
  nginx     4659       nginx mem    REG    0,1       4096     0    1188 /dev/zero
  nginx     4972       nginx mem    REG    0,1   10485760     0    1187 /dev/zero
  nginx     4972       nginx mem    REG    0,1       4096     0    1188 /dev/zero
  nginx     4972       nginx  19u   REG   8,17  137494528     0 2103873 /tmp/nginx_proxy/6/19/0000000196 (deleted)
  nginx     4972       nginx  21u   REG   8,17  596893696     0 2104973 /tmp/nginx_proxy/1/17/0000000171 (deleted)
  nginx     4972       nginx  24u   REG   8,17  298344448     0 2105098 /tmp/nginx_proxy/3/17/0000000173 (deleted)
  nginx     4972       nginx  25u   REG   8,17 1785765888     0 2105000 /tmp/nginx_proxy/2/17/0000000172 (deleted)
  nginx     4972       nginx  29u   REG   8,17  894984192     0 2105100 /tmp/nginx_proxy/4/17/0000000174 (deleted)
  nginx     4972       nginx  31u   REG   8,17 1489068032     0 2101531 /tmp/nginx_proxy/5/17/0000000175 (deleted)
  nginx     4972       nginx  35u   REG   8,17  745529344     0 2103341 /tmp/nginx_proxy/7/17/0000000177 (deleted)
  nginx     4972       nginx  37u   REG   8,17  965054464     0 2105961 /tmp/nginx_proxy/2/21/0000000212 (deleted)
  nginx     4972       nginx  41u   REG   8,17 1340678144     0 2103603 /tmp/nginx_proxy/0/18/0000000180 (deleted)
  nginx     4972       nginx  43u   REG   8,17 1633722368     0 2101619 /tmp/nginx_proxy/3/18/0000000183 (deleted)
  nginx     4972       nginx  44u   REG   8,17 1927659520     0 2103872 /tmp/nginx_proxy/5/19/0000000195 (deleted)
  nginx     4972       nginx  52u   REG   8,17  795958957     0 2103671 /tmp/nginx_proxy/2/19/0000000192 (deleted)
  nginx     4972       nginx  53u   REG   8,17 1591967405     0 2103702 /tmp/nginx_proxy/3/19/0000000193 (deleted)
  nginx     4972       nginx  58u   REG   8,17 2387853312     0 2103313 /tmp/nginx_proxy/1/21/0000000211 (deleted)

[nix-shell:~]# lsof +L1 | awk '/nginx/ {sum += $7} END {print sum/1024/1024/1024 " GiB"}'
  14.5528 GiB

Remember proxy_max_temp_file_size? Let’s read the documentation more carefully this time:

When buffering of responses from the proxied server is enabled, […], a part of the response can be saved to a temporary file.

The zero value disables buffering of responses to temporary files.

Nginx is buffering the 2.2GB file my program is serving to temporary files. Oh dear. Let’s fix that:

    "<...>.kanjideck.com" = base {
      "/" = {
        proxyPass = "http://127.0.0.1:" + toString(ports.kanjideck-fulfillment) + "/";
        extraConfig = ''
          proxy_buffering off;
          proxy_max_temp_file_size 0;
        '';
      };
    };

Grafana immediately cheered up, the server was finally fresh and lean and disk usage jumped to 20% with no more spikes:

In the disk usage graph images you can find the sudden drop to acceptable levels, which now reigns.

4 Conclusion

• The server couldn’t serve access requests from 20:40 to sometime around 23:00, i.e. for about the first 2 hours immediately after launch.

• Secondly, users couldn’t download the large file despite the remaining ones being available.

• Both of these bugs turned out to be misconfigurations in the nginx reverse proxy.

• It’s difficult to reason under pressure. Experience, that I didn’t have here, would have helped.

Note: this was written fully by me, human.

• 1 The initial idea (August 2024)
  • 1.1 What if ?
• 2 Manufacturing physical cards (September 2024)
• 3 Starting a Company in the U.S. (October 2024)
  • 3.1 Accounting and Taxes
• 4 Spreadsheets and Pricing (November 2024)
• 5 Digital Infrastructure (December 2024)
• 6 Marketing and Ads (January 2025)
• 7 Burn-out (March 2025)
• 8 Reaching out for help (October 2025)
• 9 Launching on Kickstarter (January 2026)
  • 9.1 Murphy’s Law
• 10 Conclusion

I think this is my first non-Haskell related post on this blog! This time, I want to walk through how I successfully launched Kanjideck on Kickstarter starting from zero, in my spare time.

I want to go over the initial side project, how that turned into a more ambitious idea, manufacturing and testing a physical product, setting up a company, spreadsheets, setting up the digital infrastructure for the business, marketing and ads, burn-out, launching, and reaching out for help as a solo entrepreneur.

1 The initial idea (August 2024)

In the summer of 2024, with nothing to do after having finished reading “Babel” by R. F. Kuang (a book which happened to be about language and etymology and that I thoroughly enjoyed), I decided to get back to learning Japanese.

Japanese is a challenging language to learn (at least in the West). Mostly, that’s due to Kanji, one of the three types of characters used in written Japanese. Hiragana and Katakana are phonetic scripts, where to each symbol corresponds exactly one sound, and there are about 50 of each (these can be learnt in about a week). On the other hand, there are about 2,136 Kanji needed for fluency¹. Each Kanji is a (potentially complex) symbol with one or more readings and one or more meanings. Anyway, that means there are a lot of Kanji to memorize.

I’m a big fan of spaced repetition and Anki. If you don’t know about spaced repetition, see “How to Remember Anything Forever-ish”. I had tried to study Kanji seriously in the past, but the resources I was using at the time, despite using them consistently (in Anki), weren’t doing it for me. Perhaps surprisingly, there is a big design space for Kanji learning resources. One particularly annoying aspect of many resources is the use of mnemonics, which appeal to brute-force more so than understanding. To me, they felt distracting and misleading, where a completely made up story about the distinctive features of the character was used rather than appealing to the much more useful etymological origins of the character.

I guess I have a lot to say about why I think etymology-based learning for Kanji is so useful, how it makes understanding Kanji possible (alongside memorising them), and how this understanding makes learning new ones that much easier. However, to stay on track, I’ll just refer those curious to the Kanjideck Guide to read more about its design².

Resuming Japanese in 2024, I wanted to program my own Anki deck, compiling just the right information for each Kanji card. I knew exactly what information I found valuable, so it was just a matter of finding the right sources and conjuring up the right HTML (Anki cards are rendered using HTML). I had a working prototype within 2 days, and it looked gorgeous³:

Fig 1. Kanji: Talk	Fig 2. Kanji: Past

It looked so nice, in fact, that I thought “what if”?

1.1 What if ?

The digital Anki deck was working fantastically well and the design turned out beautiful. I wanted to print some cards into the physical realm and experiment studying with a physical resource⁴. Namely, I wanted to try the Leitner System, a well-known system for physical spaced repetition.

I thought that printing custom cards would require a lot of upfront orders (this turned out to be false, see next section), and started entertaining the idea of making the resource I had created more widely available and start a small business⁵.

I had heard about Kickstarter in passing – a crowd-funding platform which allows people to raise money in pre-orders, but only charges customers if there are enough pre-orders to kickstart the business.

I then decided I’d run a Kickstarter to sell my Japanese Kanji deck in both physical and digital versions, and gather enough up-front orders to be able to print the decks out at a reasonable price per unit.

I just needed a physical version of Kanjideck first! And a project title: Kanjideck.

2 Manufacturing physical cards (September 2024)

I called a few local printing companies (location: Portugal), when I was first trying to figure out how to print the cards. Without knowing exactly who to call, none of those I contacted were able to do playing cards specifically, and required bulk orders in any case.

Eventually, I stumbled upon MakePlayingCards.com, a printing business specialized in playing cards from whom I could buy a single copy of a custom deck, and easily consult the prices of bulk orders. They do worldwide shipping and fulfillment, mentioning fulfillment for Kickstarter by name.

Based on the digital version of Kanjideck I made for Anki using HTML, I programatically generated a PDF per card with a headless browser through Playwright, and converted each card from PDF to a high-resolution PNG (fit for printing) using ImageMagick.

I ordered a test copy of 60 Kanjideck cards using their single-copy custom print feature on September 10th. That was about 15€ at the time.

While waiting for the cards to arrive, I started learning the 3D modeling tool Blender. I had seen many Kickstarter videos which used pretty amazing animations to showcase their product. I was decided to attempt this myself. A few days in I was able to model the cards and render them in some fairly OK scenes:

Fig 5. Blender render	Fig 6. Blender render

I kept improving with time, but eventually realized I was not going to be able to do an animated video which looked good enough to serve as the video showcasing the project on the Kickstarter page.

In the meantime, the cards arrived from the manufacturer. These were the very first prototype:

Fig 7. First prototype	Fig 8. First prototype6

I kept daily driving the Anki deck, improving the cards information and layout, and designing the product. Two other important things I tackled at this time were:

• Meta cards for the physical deck, including a tutorial, reference cards for Hiragana and Katakana, a reference card for verb and adjective conjugation, and markers for physical spaced repetition (to make a kind of Leitner box using them to separate which cards are in which level)

• The box design to package the cards. The Blender skills I had developed ended up being very helpful when designing the deck boxes, as I was able to lay out my design on a 3D model with accurate dimensions and preview it.

The second order, on October 19th, included three copies of the same deck of cards, properly packaged, using various different sizes/textures for the cards and for the boxes. The price for printing each single deck of 90 cards with a rigid box was close to 50€, without shipping. The values are much more reasonable when buying at scale, but for that I need the backing through Kickstarter:

This order proved invaluable to really get a feel for the different types available and choose the best fit for the Japanese-style cards. The linen material came on top for the cards, but not for the box. Here are the three packages when they arrived⁷:

I iterated on the design a few more times, then tested all three physical boxes (for JLPT-5, JLPT-4, and JLPT-3 decks). Originally, I had three decks where each tier contained all cards from all previous tiers. That was a bad idea and at some point I pivoted to having each of the three decks correspond to a single JLPT level.

In the end, the final physical decks look like this:

3 Starting a Company in the U.S. (October 2024)

I was dismayed when I found out that Kickstarters can only be run from a few specific countries. Portugal’s not in that list! Kickstarter suggests one could open a business in the United States (a supported country) using Stripe Atlas. That’s what I ended up doing.

The exact details of how to setup a company through Stripe Atlas are now unfortunately forgotten⁸. I just have to say that it is remarkable how they will handle everything for you and deliver you the business legal documents after a few days. At the time, that cost $300. I also opened a US bank account with Mercury as per Stripe’s recommendation, which went (and has been) fantastically good.

(On a side note, Mercury are famously known in the Haskell circles for, well, using Haskell in production. They are also sponsoring all my work on the new and improved Haskell debugger!).

3.1 Accounting and Taxes

Having a company comes with some responsibilities. The kind which you typically delegate, unless of course you’re hell bent on doing everything yourself to keep expenses down. Notably, one has to keep books and eventually report taxes according to them (and they better be correct!).

Luckily, some years prior, I decided to keep my personal finances using a proper ledger with double-entry bookkeeping, accounting accounts, and the standard reports (balance statement, income sheet, cash flow). Mostly, what I studied applies to personal finance just as well as accounting for a simple business, so I knew roughly what to do with the books from the start and picked the rest up as I went.

Perhaps interestingly, I keep my books using Plain Text Accounting (PTA), a discipline of bookkeeping which uses plain text files and CLI-friendly software to read and operate on them. My PTA tool of choice is hledger (a great tool that also happens to be written in Haskell). That said, the transactions and postings formats are quite similar across the existing PTA tools, so the documentation and guides on accounting with a particular tool typically generalise well to PTA tools at large. I recommend Beancount’s “The Double-Entry Counting Method” for those curious about double-entry bookkeeping, Plain Text Accounting, or just both.

Secondly, the books are used to produce reports from which you should have sufficient and correct information to fill in the Tax Reports. Now, I was used to filing taxes in Portugal using the government’s official website form – many fields can come pre-filled in, and the ones you fill in yourself are validated for consistency with one another. I never thought much of it until I had to file taxes for my foreign owned LLC in the United States.

Tax reports in the U.S. are submitted by faxing the forms to a government number. Filing them feels a bit like playing “Keep Talking and Nobody Explodes”, a fantastic game where one player is given a bomb to defuse and the other a manual with (contrived) instructions on how to do so (but can’t see the bomb!)⁹. In this case, you get to see both the bomb and the instructions, but the difficulty is Nightmare¹⁰, and there are hidden bombs, and hidden instructions. Good luck! You should probably pay someone to do it.

4 Spreadsheets and Pricing (November 2024)

How do you price a product? (Really, how? Do recommend some theory if you can).

Moreover, Kickstarter projects have to set a goal for how many dollars the project must raise in pledges for it to be deemed successful. When that goal is achieved, the pledged money is charged to the clients and the “Rewards” (which you can think of like a pre-order) must be fulfilled. If the project doesn’t reach the goal, no money is charged and no Rewards are to be delivered. This is Kickstarter’s all-or-nothing funding model.

With that in mind, my challenge was to:

• Price three different products, considering discounts and bundles;
• With a manufacturing cost dependent on the variable amount of units sold of each type;
• Across the world, with variable VAT/sales tax and shipping per country.

I’m going to skip ahead to the better and refined final iteration of my spreadsheet, which I used to make the final decisions about the price and Kickstarter goal:

I’m not a spreadsheet expert by any means (though I know how to implement a poor man’s one in a single line of Haskell¹¹), so don’t take this as a necessarily good way of approaching this problem. In fact, if you know how I could’ve done this better I’ll gladly take some pointers.

My strategy was to have one row per deck type, with a different row for the early bird discounted deck (EBN5) vs. the N5 deck vs. the N5+N4 combination. Rows are grouped by regions which have similar VATs (picking a worst-case-ish approximation for the region) and the same shipping. For each deck I table in green the revenue from each sale of one unit (i.e. unit price and shipping charged) and in red the costs of selling that unit (i.e. manufacturing, shipping cost, Kickstarter fee, Stripe fee, and VAT). In blue are key derived metrics per unit (i.e. profit, profit %, and how much shipping I’m charging as a % of the unit price).

Note: In Kickstarter, there is no way to differentiate pricing per region, so some differentiation which takes into account e.g. VAT is made via the shipping charged, which, in contrast, is per country.¹²

To manage the variable number of orders per deck, I set a baseline: 250 copies of the JLPT-5 deck (90 cards), 250 copies of the JLPT-4 deck (177 cards), and 100 copies of the JLPT-3 deck (390 cards). Those amounts were sufficient to have a positive margin. The Kickstarter funding goal was derived roughly from the price charged for those orders summed. In a separate cell, I had a multiplier which I tweaked to interactively experiment and see what my margins could look like if I sold more than that baseline. FWIW, the numbers in the spreadsheet image are for the baseline exactly.

5 Digital Infrastructure (December 2024)

I got a domain kanjideck.com and, at the start of December, started hosting a handful of services on a simple Hetzner machine. The whole machine is configured from scratch using NixOS and I can build the full machine derivation on my macOS machine and then copy and apply it to the remote machine. Some services I’m now self-hosting for the business:

• Plausible, analytics for the Kanjideck website and this blog
• Listmonk, for managing mailing lists (many recommended having a mailing list of interested people to build good initial momentum when the Kickstarter launches, so I did)
• Grafana, for monitoring the status of my services
• fail2ban, after noticing just how many attempts there were to log into my mail server
• NixOS Mailserver, for self-hosting my own mail server
• scrollsent, a small custom service which “wraps” listmonk and receives the mailing-list subscribe requests instead¹³.

Wait, what?! Are you self-hosting e-mail? Is that not insane?!

TL;DR: Yes.

Originally, despite setting everything up s.t. my e-mails get 10/10 on mail-tester.com, too many e-mails went straight to spam. I kept up at it, and encouraged people on the website to confirm their subscription by checking their spam inbox. I think this improved my mail server’s deliverability, as more e-mails (as far as I could tell) were being delivered after many confirmed subscriptions.

I use it reliably to get e-mail. When sending direct e-mail using my mail client there’s no feedback whatsoever on deliverability (right?), except if they’re answered. Replying to e-mails seems OK.

But, sending promotional e-mails in bulk? I was very wrong to think I had succeeded in setting everything up perfectly and building reputation… it does not matter! See Murphy’s Law section below.

During this time, I also started working on the kanjideck.com website, on the guide to using the cards and on the Kickstarter page.

Fig 14. Landing page in December	Fig 15. Plausible analytics on website

6 Marketing and Ads (January 2025)

Marketing is challenging.

I got started with the mindset of a one-man-band who is determined to learn everything that’s necessary. I started an Instagram page to promote the project and started paying for ads shortly after to drive people to my website. The goal was to build a mailing list which would help create momentum on launch day.

Social media management is not for me (more generally, social media isn’t). Creating new content regularly to Feed The Beast wasn’t fun and the effort never really seemed to pay off. I suppose that if you’re able to create engaging content then perhaps the algorithm will reward you (you’re at its mercy!), but posting simple photographs or short videos about a resource for Japanese study never made it past 10 likes naturally.

The alternative is to pay Meta, Google, TikTok, Reddit, etc.

I should devote an entire section to just how awful it is to work with Meta Ads. It is atrociously buggy, and they try to shove AI down your throat at every single turn of the road. It is truly baffling how the Core Product of a trillion-dollar company is this bad. If you suffer through this horrendous experience, you get to watch your hard earned money being drained by Meta for often questionable results¹⁴.

The worst part is that Meta has, in my experience, the best results by far out of the alternatives. All my attempts with Google, TikTok, and Reddit were an even bigger waste of money. For instance, today I tried running an Ad on Reddit again: it was easy to setup, but it blew past my $50 budget in an hour and went on to consume $78 for a total of 47 link clicks and 0 purchases.

Spending money on ads really makes me wonder what my goal is. At least something became clear to me: regardless of the goal, this is not how I want to achieve it.

7 Burn-out (March 2025)

Marketing the project became wearisome. In between struggling to build momentum for the project, creating more media content of the cards, preparing the Kickstarter page, the money spent on ads, and the looming task of filming and editing the project video for Kickstarter, I slowly started losing interest in the project and being fatigued from making any decision about it. I missed the launch deadline I set for myself, with no video anywhere near completed.

Add to this the instability in trade caused by the Trump tariffs, considering my manufacturer was in China, I felt even less motivated to continue working on the project. On April 9th, I posted to Instagram saying Kanjideck was suspended indefinitely.

Fig 18. Received the final prototypes	Fig 19. Announced suspension few days later

8 Reaching out for help (October 2025)

I think I eventually came to the realization that I couldn’t, nor needed, to do all of this alone.

My sister’s an excellent artist. Here’s a recent painting of hers:

I floated the idea of having her film and edit the video for the Kickstarter at some point and she was happy to help me. Moreover, she would handle all of the social media content and management. That took a huge burden off of my hands, and I was suddenly free to tackle the parts of the project I was enthusiastic about (like finishing the explanations of how to study with Kanjideck, finalizing the prices, and actually launching!).

After some iterations of filming me, the cards, and a lot of time editing, in December 2025 we had a fantastic Kickstarter video for Kanjideck:

Also then, my partner started taking care of making Ad creatives (all of which were significantly better looking than mine). And my mom wrote a few draft e-mails in preparation for the launch!

(Seriously, I wouldn’t have been able to ever launch were it not for all your help!)

9 Launching on Kickstarter (January 2026)

Kanjideck launched on Kickstarter on the 27th of January.

A few days prior to the launch, I sent two e-mails announcing the date and time at which the project would be live. Backing early would ensure an early bird discount and help the project gain momentum. The funding goal is $55,000.

On launch day at 13:00 GMT+0 we sent e-mails to everyone subscribed, posted on social media, and opened the Kickstarter!

In the first hour we received about $1,500 in pledges and until the end of the first day we had raised about $5,000 in total. On the second we raised about $2,500, and on the third $1,100. That’s still far from the all-or-nothing goal! It’s certainly not clear, but I still believe it might be possible to reach the funding goal before the campaign ends (on February 26).

As I’m writing this post, the Kickstarter page displays $8,808 raised from 136 backers:

(And, a week later, we’re at $15K (28%) from 254 backers…!)

There’s of course more to do until the campaign is over, and especially more to do afterwards if the campaign is successful. But I’ll leave that for a follow up post.

In the meantime, I’ll continue studying. I’m at 1130 Kanji learned over 434 consecutive days, but there’s still about a thousand more to go!

9.1 Murphy’s Law

To go over a few of the things that did go wrong near the launch:

• Right when I sent the first round of e-mails ever to my subscribers, Google was having an incident where a bug caused misclassification of most e-mails as spam

• Remember when I said that hosting my own mail server had gone wrong? When I sent the second round of e-mails, on the day before the launch, to my 1400 subscribers, I immediately received some 300 “undelivered mail” notifications justified by Microsoft having blocklisted my mail server’s IP.

  I hastily migrated my mailing list to SendGrid to make sure I would have proper deliverability on the launch day.

• On the day I launched, Kickstarter had an outage and was down for about an hour

That sounds amusingly unlucky, but on the other hand it might just be that, when it was my product on the line, I noticed all the internet issues that I wouldn’t typically care about.

10 Conclusion

It was worth it.

1 The Haskell Debugger for GHC 9.14

The Haskell Debugger is ready to use with GHC-9.14!

The installation, configuration, and talks can be found in the official website. The tl;dr first step is installing the debugger:

$ ghc --version # MUST BE GHC 9.14
The Glorious Glasgow Haskell Compilation System, version 9.14.1

$ cabal install haskell-debugger \
    --allow-newer=base,time,containers,ghc,ghc-bignum,template-haskell \
    --enable-executable-dynamic # ON WINDOWS, DO NOT PASS --enable-executable-dynamic
...

$ ~/.local/bin/hdb --version # VERIFY IT'S THE LATEST!
Haskell Debugger, version 0.11.0.0

The second step is configuring your editor to use the debugger via the Debug Adapter Protocol (DAP). - For VSCode, install the haskell debugger extension. - For Neovim, install nvim-dap and configure it for haskell-debugger - For other editors, consult your DAP documentation and let others know how!

Bug reports and discussions are welcome in the haskell-debugger issue tracker.

My MuniHac 2025 talk also walks through the installation, usage, and design of the debugger. Do note much has been improved since the talk was given, and much more will still improve.

1.1 A little bit more info

The debugger work is sponsored by Mercury. It’s the project in which I’ve spent most of my full working days (for almost a full year now), with the invaluable help from my team at Well-Typed.

The debugger is meant to work both on trivial files and on large and complex codebases¹. It is a GHC application so all features are supported. Like HLS, it also uses hie-bios to automatically configure the session based on your cabal or stack project.

Robustness is a main goal of the debugger. If anything doesn’t work, or if you have performance issues, or something crashes, please don’t hesitate to submit a bug. We’ve got a small but respectable testsuite, and have tested performance by debugging GHC itself, but there’s much still to be fixed and improved.

Roadmap: There’s a lot to do. I’m currently working on callstacks and multi-threaded support. Do let me know what features would be most important to you, so I can also factor that into the future planning.

The extended version of the paper, which includes all proofs, is available here [arXiv, PDF, DOI].

The short-ish story: In 2023, for my Master’s thesis, I reached out to Arnaud Spiwack to discuss how Linear Types had been implemented in GHC. I wanted to research compiler optimisations made possible by linearity. Arnaud was quick to tell me:

“Well yes, but you can’t!“

“Even though Haskell is linearly typed, Core isn’t!”¹

Linearity is ignored in Core because, as soon as it’s optimised, previously valid linear programs become invalid. It turns out that traditional linear type systems are too syntactic, or strict, about understanding linearity – but Haskell, regardless of linear types, is lazily evaluated. Improving optimisations would have to wait.

Our paper presents a system which, in contrast, also accepts programs that can only be understood as linear under non-strict evaluation. Including the vast majority of optimised linear Core programs (with proofs!).

The key ideas of this paper were developed during my Master’s, but it took a few more years of on-and-off work (supported by my employer Well-Typed) with Bernardo to crystalize the understanding of a “lazy linearity” and strengthen the theoretical results.

Now, the proof of the pudding is in the eating. Go read it!

Abstract

• 1 Announcing: xcframework
  • 1.1 XCFrameworks
  • 1.2 How to install xcframework
  • 1.3 How to use the XCFramework in XCode
  • 1.4 Building simple Swift package
  • 1.5 Must use Cabal Foreign Library stanza
  • 1.6 Conclusion

I’ve written about Haskell x Swift interoperability before. Calling Haskell from Swift is about marshalling and the foreign function interface. But Creating a macOS app with Haskell and Swift tells the much messier tale of hijacking XCode to vodoo together the Haskell library, its headers, and two handfuls of other magic ingredients into one buildable SwiftUI application.

Stop! Don’t click on the last link. No, it turns out that my XCode sallies strayed very far from the yellow brick road. The IDE is confused. Recompilation bugs abound. Complexity is through the roof juggling .modulemaps, .xcconfig dynamic settings, and sketchy .sh scripts.

Let’s walk the happy path.

1 Announcing: xcframework

Perhaps obvious in retrospect, the demon-less way to add a Haskell library to the dependencies of a Swift application is to build an independent Swift Package wrapping the Haskell library – something that can be done without XCode in sight. Easy peasy:

1. Build the Haskell library using Cabal
2. Create a Swift package from the Haskell artifacts
3. Add the Swift package as a dependency to the project

And it turns out that (1) and (2) can be merged together using Cabal SetupHooks!

Moreover, I’m happy to announce I’ve neatly packaged and released that build process automation as a Haskell library called xcframework on Hackage.

Onwards! – for what it does and how to use it.

1.1 XCFrameworks

Apple introduced XCFramework bundles back in a WWDC19 session. An XCFramework is a multiplatform binary framework bundle.

For our purposes, that means we can create a Swift Package just from a binary linkable artifact and a couple of header files. Then, any Swift project can depend on this binary Swift package and call the functions exposed to the headers and make sure the bundled library will be linked in with the final executable. Specifically, the xcframework Haskell library, for a given Haskell library, bundles:

• The foreign shared library (.dylib) resulting from building with GHC/Cabal
• The foreign export headers generated from the foreign export <haskell_function> declarations
• The RTS headers
  • which are needed to initialize the RTS from Swift
  • and because they are #included by the foreign export headers
• A .modulemap exporting the foreign exported functions and HsFFI.h
  • The module map basically turns the headers into a Swift module that can be transparently imported from other Swift modules.

And any Swift library or application can transparently depend on this .xcframework and use the foreign exported Haskell functions without further ado.

1.2 How to install xcframework

In your cabal file, change the build-type to Hooks (and set cabal-version: 3.14 if not set already):

- build-type:     Simple
+ build-type:     Hooks

And add a setup-depends stanza with a dependency on xcframework:

custom-setup
  setup-depends:
    base        >= 4.18 && < 5,
    xcframework >= 0.1

Finally, create a file called SetupHooks.hs in the root of your Cabal package with the following contents, substituting the _build/MyHaskellLib.xcframework string for the filepath to where the .xcframework should be written:

module SetupHooks ( setupHooks ) where

import Distribution.XCFramework.SetupHooks

setupHooks :: SetupHooks
setupHooks = xcframeworkHooks "_build/MyHaskellLib.xcframework"

Now, whenever you run cabal build, the built libraries will also be bundled into an .xcframework.

1.3 How to use the XCFramework in XCode

In XCode:

1. Navigate to the target settings of your project.
2. Find under “General” the “Frameworks, Libraries, and Embedded Content” (or similar) section.
3. Click the add button and add the .xcframework framework outputted at the specified path by Cabal

Now, in the entry Swift module, import the RTS and init/exit the RTS. For instance, in a sample SwiftUI app:

  import SwiftUI
+ import Haskell.Foreign.Rts
  
  @main
  struct MyExample: App {
+ 
+     init() {
+         hs_init(nil, nil)
+
+         NotificationCenter.default
+           .addObserver(forName: NSApplication.willTerminateNotification,
+                        object: nil, queue: .main) { _ in
+           hs_exit()
+         }
+     }
+ 
      var body: some Scene {
          WindowGroup {
              ContentView()
          }
      }
  }

Finally, in any Swift module, do import Haskell.Foreign.Exports. For now, the name Haskell.Foreign.Exports is fixed and exports all foreign-exported functions, but it could be improved in the future (perhaps it’s a good task to contribute a patch for!)

For example, if your Haskell module looked like:

module MyLib (doSomething) where

fib :: Integral b => Int -> b
fib n = round $ phi ** fromIntegral n / sq5
  where
    sq5 = sqrt 5 :: Double
    phi = (1 + sq5) / 2

doSomething :: IO Int
doSomething = do
  putStrLn "doing some thing"
  return $ fib 42

foreign export ccall doSomething :: IO Int

In your Swift module you can now

import Haskell.Foreign.Exports

  ...
  let x = doSomething()
  ...

1.4 Building simple Swift package

The .xcframework can also be easily used in a standalone swift package built with swift build.

In your Package.swift, add MyHaskellLib.xcframework as a binary target and make it a dependency of your main target. For instance, a simple library would look like:

// swift-tools-version: 6.1
import PackageDescription

let package = Package(
    name: "MySwiftLib",
    platforms: [
        .macOS(.v15)
    ],
    products: [
        .library(name: "MySwiftLib", targets: ["MySwiftLib"])
    ],
    targets: [
        .target(name: "MySwiftLib", dependencies: ["MyHaskellLib"], path: "Swift"),
        .binaryTarget(
            name: "MyHaskellLib",
            path: "haskell/_build/MyHaskellLib.xcframework"
        )
    ]
)

Now you can use the Haskell.Foreign.Exports import in any module in the package as explained above, for instance in Swift/MySwiftLib.hs:

import Foundation
import Haskell.Foreign.Exports

public struct Fib {
  var val: Int64
}

public func mkFib() -> Fib {
    let x = doSomething()
    return Fib(val: x)
}

Build the Swift package using swift build in the project root.

1.5 Must use Cabal Foreign Library stanza

Unfortunately, while I don’t figure out how to link the right amount of things into the .xcframework after building a normal library component in Cabal, the foreign exports must be exported from a foreign-library Cabal stanza:

foreign-library myexample
    type: native-shared
    options: standalone
    other-modules:    MyLib
    build-depends:    base ^>=4.20.0.0
    hs-source-dirs:   src
    default-language: GHC2021

To clarify the instructions, I put together a small demo project with a working setup – if you want to try it out. Remember to build the Cabal library first!

1.6 Conclusion

Building the Haskell library as an independent Swift Package is a much more robust way of adding a Haskell dependency to a Swift application.

The xcframework Haskell library makes it easy to create XCFrameworks from Haskell packages by leveraging the SetupHooks very nicely designed API.

While this work further lowers the bar for integrating Haskell and Swift, marshaling and sharing high-level datatypes remains challenging. Calling Haskell from Swift explored the basics of using more interesting types across the FFI, but I’m also working on a more automated approach using TH and GHC plugins.

Finally, I’m looking forward to bug reports if you try it out.

• 1 Unsure Calculator
  • 1.1 Sampling it up
  • 1.2 Calculator Expressions
  • 1.3 Showing up
  • 1.4 Conclusion

1 Unsure Calculator

The recently trendy Unsure Calculator makes reasoning about numbers with some uncertainty just as easy as calculating with specific numbers.

The key idea is to add a new “range” operator (written ~) to the vocabulary of a standard calculator. The range x~y denotes that a real value is uncertain, but we are 95% sure that it falls between x and y¹.

Reading the notation is easy: when you see 10~15, you say: “ten to fifteen”.

Arithmetic operations and friends (e.g. sin, or log) transparently operate on ranges and literal numbers alike. Calculation results in a plot with a range of values that the input expression can take, and with what frequency.

The motivation behind the original article is neat, so I’ll just recommend you read it there to learn how and why you’d use such a calculator. Here’s a real life example they used:

1400~1700 * 0.55~0.65 - 600~700 - 100~200 - 30 - 20

Now, let’s implement it.

1.1 Sampling it up

Summon a probability monad from the void².

data Dist a where
  Return :: a -> Dist a
  Bind   :: Dist b -> (b -> Dist a) -> Dist a
  Normal :: Double -> Double -> Dist Double

instance Monad       Dist where (>>=) = Bind
instance Applicative Dist where pure  = Return; (<*>) = ap
instance Functor     Dist where fmap  = liftM

The monad instance is free: pure = Return and (>>=) = Bind. The Normal constructor denotes a normal distribution given the standard deviation and mean. With do-notation we can easily construct a complex tree mixing Returns, Binds, and Normals. For instance:

d = do
  s <- Normal 0 1
  return (5 + s)

desugars to

d = Bind (Normal 0 1) (\s -> Return (5 + s))

Then, embue meaning onto a Dist a by allowing an a to be sampled according to the distribution the Dist represents. We use StdGen from random as a source of uniform pseudo-randomness:

sample :: StdGen -> Dist a -> a
sample g d = case d of
  Return x -> x
  Normal mean std_dev -> n1 * std_dev + mean
    where ((u1, u2), _) = uniformR ((0,0), (1,1)) g
          (n1,  _)      = boxMuller u1 u2
  Bind d f -> sample g1 (f (sample g2 d))
    where (g1, g2) = splitGen g

Sampling a distribution is simple:

• For a constant x, return x
• For a normal distribution, use the Box Muller transform to transform a pair of uniformly distributed random numbers in [0, 1] into a pair of normally distributed random numbers. Then, scale according to the normal distribution parameters (mean, std_dev).
• For Bind, sample a from Dist a, feed it to the continuation, and then sample from the resulting Dist b. Do make sure the PRN generator is split.

Box-Muller is implemented very literally as:

boxMuller u1 u2 = (r * cos t, r * sin t) where r = sqrt (-2 * log u1)
                                               t = 2 * pi * u2

We can try sampling the example above:

> sample (mkStdGen 3) (Bind (Normal 0 1) (\s -> Return (5+s)))
5.079952851990287

And we can also use sequence . repeat to generate an infinite list of samples:

> take 5 $ sample (mkStdGen 4) $ sequence $ repeat $ (Bind (Normal 0 1) (\s -> Return (5+s)))
[4.133071135842417,5.270868355973887,6.9698645379055355,3.8369680071523447,4.2986629449038425]

1.2 Calculator Expressions

Conjure up an eDSL for calculator expressions:

data Expr
  = Num Double
  | Add Expr Expr   | Mul Expr Expr
  | Abs Expr        | Signum Expr
  | Negate Expr     | Div Expr Expr
  | Exp Expr        | Log Expr
  | Sin Expr        | Cos Expr
  | Range Expr Expr

(~) = Range

instance Num Expr where
  (+) = Add; (*) = Mul
  negate = Negate; abs = Abs
  signum = Signum; fromInteger = Num . fromInteger

instance Fractional Expr where
  (/) = Div; fromRational = Num . fromRational

instance Floating Expr where
  pi = Num pi; exp = Exp; log = Log; sin = Sin; cos = Cos

By defining ~ to be Range and overloading math we can immediately write

1400~1700 * 0.55~0.65 - 600~700

to mean

Add (Mul (Range 1400 1700) (Range 0.55 0.65)) (Negate (Range 600 700)))

Breathe life into expressions by allowing them to be evaluated to a distribution that can later be sampled. This is where the Functor, Applicative, and Monad instances for Dist come into play:

eval :: Expr -> Dist Double
eval e = case e of
  Num d       -> return d
  Add e1 e2   -> (+) <$> eval e1 <*> eval e2
  Mul e1 e2   -> (*) <$> eval e1 <*> eval e2
  Negate e    -> negate <$> eval e
  Abs e       -> abs <$> eval e
  Signum e    -> signum <$> eval e
  Div e1 e2   -> (/) <$> eval e1 <*> eval e2
  Exp e       -> exp <$> eval e
  Log e       -> log <$> eval e
  Sin e       -> sin <$> eval e
  Cos e       -> cos <$> eval e
  Range e1 e2 -> do
    a <- eval e1
    b <- eval e2
    let mean = (a + b) / 2
        std_dev = (b - a) / 4
    Normal mean std_dev

Most cases just recursively eval the sub-expressions into Dist Doubles and then lift a math operation on Doubles into the probability monad. The interesting case is Range, which uses the evaluated sub-expressions, referring to the lower and upper range bound, to compute the standard deviation and mean of the normal distribution – a Dist is returned by applying the “primitive” Normal constructor accordingly.

We can take a few samples of the first example in this post:

> take 10 $ sample (mkStdGen 5) $ sequence $ repeat $ eval (1400~1700 * 0.55~0.65 - 600~700 - 100~200 - 30 - 20)
[8.808397505000045,-20.049171720836654,12.753715226577157,-22.77067243260501,-70.3671725933005,-11.610935890955716,157.612422580839,75.61254061496092,88.04359089198991,-37.280109436085866]

Even though we are taking samples from the distribution constructed by that compound expression, we cannot make too much sense of these numbers. We need to plot them in a histogram, just like the original calculator does, to make this useful.

1.3 Showing up

I’ll spare the nitty-gritty details, but, lastly, we define a Show instance for Expr that plots a number of samples from the distribution resulting from the given expression. In order:

• Eval the expression e into a Dist Double
• Take 25000 samples
• Group each sample into the interval it is closest to (in collapseIntervals)
• Compute the ratio of samples in each box over the total number of samples, to display as a percentage
• For each box, plot a line (in line and showRow)

instance Show Expr where
  show e = concatMap showRow intervals where
    samples = take 25000 . sample (mkStdGen 0) . sequence . repeat . eval
    intervals = collapseIntervals 40 (samples e)
    line p = replicate spaces ' ' ++ replicate normalized ':'
      where spaces = 35 - normalized
            normalized = round ((p/max_prob) * 30)
            max_prob = maximum $ map snd intervals
    showRow (elem, prob) = line prob ++ " | " ++ printf "%.1f (%.1f" elem (prob*100) ++ "%)\n"

collapseIntervals :: Int -> [Double] -> [(Double, Double)]
collapseIntervals n (sort -> samples) =
  let (low, high) = (head samples, last samples)
      step  = (high - low) / fromIntegral n
      boxes = [low, low+step .. high] 
      total = fromIntegral (length samples)
   in M.toList $ M.map (/total) $ M.fromListWith (+)
        [ (minimumBy (compare `on` \box -> abs (box - s)) boxes, 1)
        | s <- samples ]

Let’s try to print out the first example now, by simply typing in GHCi:

> 1400~1700 * 0.55~0.65 - 600~700 - 100~200 - 30 - 20

                                    | -188.2 (0.0%)
                                    | -162.2 (0.0%)
                                    | -149.2 (0.0%)
                                    | -136.1 (0.1%)
                                    | -123.1 (0.1%)
                                    | -110.1 (0.1%)
                                  : | -97.1 (0.2%)
                                 :: | -84.1 (0.4%)
                                 :: | -71.1 (0.6%)
                               :::: | -58.1 (1.1%)
                            ::::::: | -45.1 (1.7%)
                            ::::::: | -32.1 (1.8%)
                        ::::::::::: | -19.0 (2.9%)
                    ::::::::::::::: | -6.0 (3.8%)
                  ::::::::::::::::: | 7.0 (4.3%)
              ::::::::::::::::::::: | 20.0 (5.3%)
          ::::::::::::::::::::::::: | 33.0 (6.1%)
        ::::::::::::::::::::::::::: | 46.0 (6.7%)
     :::::::::::::::::::::::::::::: | 59.0 (7.4%)
     :::::::::::::::::::::::::::::: | 72.0 (7.5%)
     :::::::::::::::::::::::::::::: | 85.0 (7.5%)
       :::::::::::::::::::::::::::: | 98.0 (7.0%)
         :::::::::::::::::::::::::: | 111.1 (6.6%)
            ::::::::::::::::::::::: | 124.1 (5.8%)
              ::::::::::::::::::::: | 137.1 (5.3%)
                  ::::::::::::::::: | 150.1 (4.3%)
                      ::::::::::::: | 163.1 (3.3%)
                       :::::::::::: | 176.1 (2.9%)
                          ::::::::: | 189.1 (2.2%)
                            ::::::: | 202.1 (1.7%)
                              ::::: | 215.1 (1.2%)
                                ::: | 228.2 (0.8%)
                                 :: | 241.2 (0.6%)
                                  : | 254.2 (0.3%)
                                  : | 267.2 (0.2%)
                                    | 280.2 (0.1%)
                                    | 293.2 (0.0%)
                                    | 306.2 (0.0%)
                                    | 319.2 (0.0%)
                                    | 332.2 (0.0%)

Success! This looks very similar to the original histogram for the same expression. Let’s also try Drake’s equation as described in the original article:

> 1.5~3 * 0.9~1.0 * 0.1~0.4 * 0.1~1.0 * 0.1~1.0 * 0.1~0.2 * 304~10000

                                    | -307.0 (0.0%)
                                    | -193.4 (0.0%)
                                    | -155.5 (0.0%)
                                    | -117.7 (0.0%)
                                    | -79.8 (0.1%)
                                  : | -42.0 (0.6%)
                       :::::::::::: | -4.1 (8.3%)
     :::::::::::::::::::::::::::::: | 33.7 (20.9%)
        ::::::::::::::::::::::::::: | 71.6 (19.1%)
              ::::::::::::::::::::: | 109.4 (14.6%)
                    ::::::::::::::: | 147.3 (10.5%)
                        ::::::::::: | 185.2 (7.4%)
                           :::::::: | 223.0 (5.3%)
                              ::::: | 260.9 (3.6%)
                               :::: | 298.7 (2.6%)
                                ::: | 336.6 (1.9%)
                                 :: | 374.4 (1.4%)
                                  : | 412.3 (0.9%)
                                  : | 450.1 (0.7%)
                                  : | 488.0 (0.5%)
                                    | 525.9 (0.3%)
                                    | 563.7 (0.3%)
                                    | 601.6 (0.2%)
                                    | 639.4 (0.1%)
                                    | 677.3 (0.1%)
                                    | 715.1 (0.1%)
                                    | 753.0 (0.1%)
                                    | 790.8 (0.0%)
                                    | 828.7 (0.0%)
                                    | 866.5 (0.0%)
                                    | 904.4 (0.0%)
                                    | 942.3 (0.0%)
                                    | 980.1 (0.0%)
                                    | 1018.0 (0.0%)
                                    | 1055.8 (0.0%)
                                    | 1093.7 (0.0%)
                                    | 1169.4 (0.0%)
                                    | 1207.2 (0.0%)

That’s it!

1.4 Conclusion

A few final remarks:

• This is a neat and simple implementation. It highlights some of Haskell’s strengths in an interesting task.
• Since our calculator implementation is embeded in Haskell, we can leverage the GHCi REPL – most importantly, we can do variable binding and define functions with no additional implementation cost.
• You can try the full program (exactly 100 lines) on your machine by loading it with cabal repl <file.hs> and typing unsure expressions at will.

• 1 A workout planner in 100 lines of Haskell

I have recently started doing some outdoors bodyweight workouts. I also want to start running again, but I’m recovering from a minor knee injury until the start of next month.

Tonight I decided to put together a weekly schedule to start following next month. The first pen and paper versions were fine, but I wasn’t completely satisfied. The next logical step was to write a quick program to see what possible plans I was missing.

The schedule must satisfy a few constraints, but the core of it is that I should do, every week, on one axis, one short run (high-intensity) and one long run (long distance), and, on the other axis, have two pull days (as in pull-ups), two push days (as in push-ups), and two leg days (as in squats).

Finding a weekly workout that satisfies certain constraints is an answer-set-programming kind of problem, best solved by some kind of logic programming. Rather than turning to Prolog or Clingo, I decided to just stick to Haskell and use the logic-programming monad from logict!

1 A workout planner in 100 lines of Haskell

What follows is mostly just the demonstration of using logict applied to this particular problem. I believe the weeklySchedule function can be easily understood in general, even by anyone unfamiliar with Haskell and/or logic programming – and that’s the meat of this short post and program.

Note that the program is a cabal shell script which can be run by executing the script file (as in ./ScheduleExercise, as long as cabal is in path). It is standalone, and exactly 100 lines (with comments, shebangs and everything). Feel free to try and modify it!

#!/usr/bin/env cabal
{- cabal:
build-depends: base, logict
-}

import Control.Applicative
import Control.Monad
import Control.Monad.Logic
import Data.List
import Data.Maybe

workout = ["Push day", "Pull day", "Leg day", "No workout"]
running = ["Long run", "Short run", "No run"]
weekdays = ["Seg.", "Ter.", "Qua.", "Qui.", "Sex.", "Sab.", "Dom."]

weeklySchedule :: Logic [(String, [String])]
weeklySchedule = do
  -- For every weekday, pick an element from `workout` and one from `running`
  -- that satisfy the "nested" conditions
  p <- forM weekdays $ \d -> do
    w <- choose workout
    r <- choose running

    -- No running on leg day
    w == "Leg day" ==> r == "No run"

    -- Short intervals run is after an outdoor pull/push workout
    r == "Short run" ==> w /= "No workout"

    -- Workout on Monday outdoors always, not legs
    d == "Seg." ==> w /= "No workout"
    d == "Seg." ==> w /= "Leg day"

    -- Pull day during the week?
    w == "Pull day" ==> (d /= "Sab." && d /= "Dom.")

    pure [w,r]

  -- Now, pick the set `p` of (weekdays X exercises) that satisfy the following conditions:

  -- One long run, one short run
  exactly 1 "Long run" p
  exactly 1 "Short run" p

  -- Two push, two pull, two leg
  exactly 2 "Push day" p
  exactly 2 "Pull day" p
  exactly 2 "Leg day" p

  -- Long run on weekend
  onDay "Long run" "Sab." p <|> onDay "Long run" "Dom." p

  -- Run spaced out at least 2 days
  daysBetween 2 "Short run" "Long run" p
  daysBetween 2 "Long run" "Short run" p

  -- Space out workouts at least 2 days
  spacedOut "Push day" 2 p
  spacedOut "Pull day" 2 p
  spacedOut "Leg day" 2 p

  -- No leg day before short run
  daysBetween 1 "Leg day" "Short run" p
  -- No leg day before a long run
  daysBetween 1 "Leg day" "Long run"  p

  -- At least one of the runs without a leg day after please
  daysBetween 1 "Short run" "Leg day" p <|> daysBetween 1 "Long run" "Leg day" p

  return (zip weekdays p)

--------------------------------------------------------------------------------
-- Logic utils

choose = foldr ((<|>) . pure) empty

exactly n s p = guard (length (filter (s `elem`) p) == n)

onDay s d p = guard (s `elem` getDay d p) where
  getDay s p = p !! fromMaybe undefined (elemIndex s weekdays)

-- space out as at least n days
spacedOut a n p = guard (all (>n) dists) where
  dists = zipWith (-) (drop 1 is) is
  is = findIndices (a `elem`) (take 14 $ cycle p {- cycle week around -})

daysBetween n a b p =
  forM_ (take 14 (cycle p) `zip` [1..]) $ \(x,i) -> do
    a `elem` x ==> all (b `notElem`) (take n $ drop i (take 14 (cycle p)))

(==>) a b = guard (not a || b)
infixr 0 ==>

--------------------------------------------------------------------------------
-- Main

printSched = mapM (\(d, ls) -> putStrLn (d ++ " " ++ intercalate ", " ls))

main = do let r = observeAll weeklySchedule
          mapM (const (putStrLn "") <=< printSched) r

The heavy lifting is done by the logict package. It allows us to consider alternative values as solutions to logic statements. The possible alternatives are separated by <|>, and observeAll returns a list with all valid alternatives. A trivial example:

x = pure "a" <|> pure "b" <|> pure "c"
print (observeAll x)

Result: ["a", "b", "c"]

If the alternative is empty, it is not an answer and therefore not included in the result. The guard combinator takes a boolean and returns empty if it is false, therefore constraining the solution.

x = pure "a" <|> empty <|> pure "c"
print (observeAll x)

Result: ["a", "c"]

Finally, we can use the monadic do notation to combine sets of alternatives. Put together with guard, we can write slightly more interesting programs:

x = do
  y <- pure "a" <|> pure "b" <|> pure "c"
  z <- pure "a" <|> pure "b" <|> pure "c"
  guard (y /= z)
  return (y,z)
print (observeAll x)

Result: [("a","b"),("a","c"),("b","a"),("b","c"),("c","a"),("c","b")]

All in all, the body of weeklySchedule uses just these principles plus a few domain-specific combinators I wrote in the Logic utils section of the code (which themselves use also only these principles from logict, plus some laziness-coolness). The program entry point (main) prints out the schedule.

And by the way, these are the workout schedules satisfying all those constraints:

Seg. Pull day, No run
Ter. Push day, Short run
Qua. No workout, No run
Qui. Leg day, No run
Sex. Pull day, No run
Sab. Push day, Long run
Dom. Leg day, No run

Seg. Pull day, No run
Ter. Leg day, No run
Qua. Push day, No run
Qui. Pull day, Short run
Sex. Leg day, No run
Sab. Push day, No run
Dom. No workout, Long run

Seg. Pull day, No run
Ter. Leg day, No run
Qua. Push day, No run
Qui. Pull day, Short run
Sex. Leg day, No run
Sab. No workout, No run
Dom. Push day, Long run

• 1 Introduction
• 2 Marshaling Inputs and Outputs
  • 2.1 Haskell’s Perspective
  • 2.2 Swift’s Perspective
• 3 Metaprogramming at the boundaries
  • 3.1 Haskell’s perspective
  • 3.2 Swift’s perspective
• 4 Remarks

This is the second installment of the in-depth series of blog-posts on developing native macOS and iOS applications using both Haskell and Swift/SwiftUI. This post covers how to call (non-trivial) Haskell functions from Swift by using a foreign function calling-convention strategy similar to that described by Calling Purgatory from Heaven: Binding to Rust in Haskell that requires argument and result marshaling.

You may find the other blog posts in this series interesting:

1. Creating a macOS app with Haskell and Swift

The series of blog posts is further accompanied by a github repository where each commit matches a step of this tutorial. If in doubt regarding any step, check the matching commit to make it clearer.

This write-up has been cross-posted to Well-Typed’s Blog.

1 Introduction

We’ll pick up from where the last post ended – we have set up an XCode project that includes our headers generated from Haskell modules with foreign exports and linking against the foreign library declared in the cabal file. We have already been able to call a very simple Haskell function on integers from Swift via Haskell’s C foreign export feature and Swift’s C interoperability.

This part concerns itself with calling idiomatic Haskell functions, which typically involve user-defined datatypes as inputs and outputs, from Swift. Moreover, these functions should be made available to Swift transparently, such that Swift calls them as it does other idiomatic functions, with user defined structs and classes.

For the running example, the following not-very-interesting function will suffice to showcase the method we will use to expose this function from Haskell to Swift, which easily scales to other complex data types and functions.

data User
  = User { name :: String
         , age  :: Int
         }

birthday :: User -> User
birthday user = user{age = user.age + 1}

The Swift side should wrap Haskell’s birthday:

struct User {
    let name: String
    let age: Int
}

// birthday(user: User(name: "Anton", age: 33)) = User(name: "Anton", age: 34)
func birthday(user: User) -> User {
    // Calls Haskell function...
}

To support this workflow, we need a way to convert the User datatype from Haskell to Swift, and vice versa. We are going to serialize (most) inputs and outputs of a function. Even though the serialization as it will be described may seem complex, it can be automated with Template Haskell and Swift Macros and packed into a neat interface – which I’ve done in haskell-swift.

As a preliminary step, we add the User data type and birthday function to haskell-framework/src/MyLib.hs, and the Swift equivalents to SwiftHaskell/ContentView.swift from the haskell-x-swift-project-steps example project.

2 Marshaling Inputs and Outputs

Marshaling the inputs and outputs of a function, from the Swift perspective, means to serialize the input values into strings, and receive the output value as a string which is then decoded into a Swift value. The Haskell perspective is dual.

Marshaling/serializing is a very robust solution to foreign language interoperability. While there is a small overhead of encoding and decoding at a function call, it almost automatically extends to, and enables, all sorts of data to be transported across the language boundary, without it being vulnerable to compiler implementation details and memory representation incompatibilities.

We will use the same marshaling strategy that Calling Purgatory from Heaven: Binding to Rust in Haskell does. In short, the idiomatic Haskell function is wrapped by a low-level one which deserializes the Haskell values from the argument buffers, and serializes the function result to a buffer that the caller provides. More specifically,

• For each argument of the original function, we have a Ptr CChar and Int – a string of characters and the size of that string (a.k.a CStringLen)

• For the result of the original function, we have two additional arguments, Ptr CChar and Ptr Int – an empty buffer in memory, and a pointer to the size of that buffer, both allocated by the caller.

• For each argument, we parse the C string into a Haskell value that serves as an argument to the original function.

• We call the original function

• We overwrite the memory location containing the original size of the buffer with the required size of the buffer to fit the result (which may be smaller or larger than the actual size). If the buffer is large enough we write the result to it.

• From the Swift side, we read the amount of bytes specified in the memory location that now contains the required size. If it turns out that the required size is larger than the buffer’s size, we need to retry the function call with a larger buffer.

  • This means we might end up doing the work twice, if the original buffer size is not big enough. Some engineering work might allow us to re-use the result, but we’ll stick with retrying from scratch for simplicity.

We will use JSON as the serialization format: this choice is motivated primarily by convenience because Swift can derive JSON instances for datatypes out of the box (without incurring in extra dependencies), and in Haskell we can use aeson to the same effect. In practice, it could be best to use a format such as CBOR or Borsh which are binary formats optimised for compactness and serialization performance.

2.1 Haskell’s Perspective

Extending the User example requires User to be decodable, which can be done automatically by adding to the User declaration:

deriving stock Generic
deriving anyclass (ToJSON, FromJSON)

With the appropriate extensions and importing the necessary modules in MyLib:

{-# LANGUAGE DerivingStrategies, DeriveAnyClass #-}

-- ...

import GHC.Generics
import Data.Aeson

The MyForeignLib module additionally must import

import Foreign.Ptr
import Foreign.Storable
import Foreign.Marshal
import Data.Aeson
import Data.ByteString
import Data.ByteString.Unsafe

Now, let’s (foreign) export a function c_birthday that wraps birthday above in haskell-framework/flib/MyForeignLib.hs, using the described method.

First, the type definition of the function receives the buffer with the User argument, and a buffer to write the User result to. We cannot use tuples because they are not supported in foreign export declarations, but the intuition is that the first two arguments represent the original User input, and the two latter arguments represent the returned User.

c_birthday :: Ptr CChar -> Int -> Ptr CChar -> Ptr Int -> IO ()

Then, the implementation – decode the argument, encode the result, write result size to the given memory location and the result itself to the buffer, if it fits.

c_birthday cstr clen result size_ptr = do

We transform the (Ptr CChar, Int) pair into a ByteString using unsafePackCStringLen, and decode a User from the ByteString using decodeStrict:

  -- (1) Decode C string
  Just user <- decodeStrict <$> unsafePackCStringLen (cstr, clen)

We apply the original birthday function to the decoded user. In our example, this is a very boring function, but in reality this is likely a complex idiomatic Haskell function that we want to expose to.

  -- (2) Apply `birthday`
  let user_new = birthday user

We encode the new_user :: User as a ByteString, and use unsafeUseAsCStringLen to get a pointer to the bytestring data and its length. Finally, we get the size of the result buffer, write the actual size of the result to the given memory location, and, if the actual size fits the buffer, copy the bytes from the bytestring to the given buffer.

  -- (3) Encode result
  unsafeUseAsCStringLen (toStrict $ encode user_new) $ \(ptr,len) -> do

    -- (3.2) What is the size of the result buffer?
    size_avail <- peek size_ptr

    -- (3.3) Write actual size to the int ptr.
    poke size_ptr len

    -- (3.4) If sufficient, we copy the result bytes to the given result buffer
    if size_avail < len
       then do
         -- We need @len@ bytes available
         -- The caller has to retry
         return ()
       else do
         moveBytes result ptr len

If the written required size is larger than the given buffer, the caller will retry.

Of course, we must export this as a C function.

foreign export ccall c_birthday :: Ptr CChar -> Int -> Ptr CChar -> Ptr Int -> IO ()

This makes the c_birthday function wrapper available to Swift in the generated header and at link time in the dynamic library.

2.2 Swift’s Perspective

In Swift, we want to be able to call the functions exposed from Haskell via their C wrappers from a wrapper that feels idiomatic in Swift. In our example, that means wrapping a call to c_birthday in a new Swift birthday function.

In ContentView.swift, we make User JSON-encodable/decodable by conforming to the Codable protocol:

struct User: Codable {
    // ...
}

Then, we implement the Swift side of birthday which simply calls c_birthday – the whole logic of birthday is handled by the Haskell side function (recall that birthday could be incredibly complex, and other functions exposed by Haskell will indeed be).

func birthday(user: User) -> User {
    // ...
}

Note: in the implementation, a couple of blocks have to be wrapped with a do { ... } catch X { ... } but I omit them in this text. You can see the commit relevant to the Swift function wrapper implementation in the repo with all of these details included.

First, we encode the Swift argument into JSON using the Data type (plus its length) that will serve as arguments to the foreign C function.

let enc = JSONEncoder()
let dec = JSONDecoder()

var data: Data = try enc.encode(user)
let data_len = Int64(data.count)

However, a Swift Data value, which represents the JSON as binary data, cannot be passed directly to C as a pointer. For that, we must use withUnsafeMutableBytes to get an UnsafeMutableRawBufferPointer out of the Data – that we can pass to the C foreign function. withUnsafeMutableBytes receives a closure that uses an UnsafeMutableRawBufferPointer in its scope and returns the value returned by the closure. Therefore we can return the result of calling it on the user Data we encoded right away:

return data.withUnsafeMutableBytes { (rawPtr: UnsafeMutableRawBufferPointer) in
    // here goes the closure that can use the raw pointer,
    // the code for which we describe below
}

We allocate a buffer for the C foreign function to insert the result of calling the Haskell function, and also allocate memory to store the size of the buffer. We use withUnsafeTemporaryAllocation to allocate a buffer that can be used in the C foreign function call. As for withUnsafeMutableBytes, this function also takes a closure and returns the value returned by the closure:

// The data buffer size
let buf_size = 1024048 // 1024KB

// A size=1 buffer to store the length of the result buffer
return withUnsafeTemporaryAllocation(of: Int.self: 1) { size_ptr in
    // Store the buffer size in this memory location
    size_ptr.baseAddress?.pointee = buf_size

    // Allocate the buffer for the result (we need to wrap this in a do { ...} catch for reasons explained below)
    do {
        return withUnsafeTemporaryAllocation(byteCount: buf_size, alignment:1) { res_ptr in

            // Continues from here ...
        }
    } catch // We continue here in due time ...
}

We are now nested deep within 3 closures: one binds the pointer to the argument’s data, the other the pointer to the buffer size, and the other the result buffer pointer. This means we can now call the C foreign function wrapping the Haskell function:

c_birthday(rawPtr.baseAddress, data_len, res_ptr.baseAddress, size_ptr.baseAddress)

Recalling that the Haskell side will update the size pointed to by size_ptr to the size required to serialize the encoded result, we need to check if this required size exceeds the buffer we allocated, or read the data otherwise:

if let required_size = size_ptr.baseAddress?.pointee {
    if required_size > buf_size {
        // Need to try again
        throw HsFFIError.requiredSizeIs(required_size)
    }
}

return dec.decode(User.self, from: Data(bytesNoCopy: res_ptr.baseAddress!,
                    count: size_ptr.baseAddress?.pointee ?? 0, deallocator: .none))

where HsFFIError is a custom error defined as

enum HsFFIError: Error {
    case requiredSizeIs(Int)
}

We must now fill in the catch block to retry the foreign function call with a buffer of the right size:

} catch HsFFIError.requiredSizeIs(let required_size) {
    return withUnsafeTemporaryAllocation(byteCount: required_size, alignment:1)
    { res_ptr in
        size_ptr.baseAddress?.pointee = required_size
        c_birthday(rawPtr.baseAddress, data_len, res_ptr.baseAddress, size_ptr.baseAddress)

        return dec.decode(User.self, from: Data(bytesNoCopy: res_ptr.baseAddress!,
                    count: size_ptr.baseAddress?.pointee ?? 0, deallocator: .none))
    }
}

That seems like a lot of work to call a function from Haskell! However, despite this being a lot of code, not a whole lot is happening: we simply serialize the argument, allocate a buffer for the result, and deserialize the result into it. In the worst case, if the serialized result does not fit (the serialized data has over 1M characters), then we naively compute the function a second time (it should not be terribly complicated to avoid this work by caching the result and somehow resuming the serialization with the new buffer). Furthermore, there is a lot of bureocracy in getting the raw pointers to send off to Haskell land – the good news is that all of this can be automated away behind automatic code generation with Template Haskell and Swift Macros.

func birthday (user : User) -> User {
    let enc = JSONEncoder()
    let dec = JSONDecoder()
    do {
        var data : Data = try enc.encode(user)
        let data_len = Int64(data.count)
        return try data.withUnsafeMutableBytes { (rawPtr:UnsafeMutableRawBufferPointer) in

            // Allocate buffer for result
            let buf_size = 1024000

            return try withUnsafeTemporaryAllocation(of: Int.self, capacity: 1) { size_ptr in
                size_ptr.baseAddress?.pointee = buf_size

                do {
                    return try withUnsafeTemporaryAllocation(byteCount: buf_size, alignment: 1) {
 res_ptr in

                        c_birthday(rawPtr.baseAddress, data_len, res_ptr.baseAddress, size_ptr.baseAddress)

                        if let required_size = size_ptr.baseAddress?.pointee {
                            if required_size > buf_size {
                                throw HsFFIError.requiredSizeIs(required_size)
                            }
                        }
                        return try dec.decode(User.self, from: Data(bytesNoCopy: res_ptr.baseAddress!, count: size_ptr.baseAddress?.pointee ?? 0, deallocator: .none))
                    }
                } catch HsFFIError.requiredSizeIs(let required_size) {
                    print("Retrying with required size: \(required_size)")
                    return try withUnsafeTemporaryAllocation(byteCount: required_size, alignment:
 1) { res_ptr in
                        size_ptr.baseAddress?.pointee = required_size

                        c_birthday(rawPtr.baseAddress, data_len, res_ptr.baseAddress, size_ptr.baseAddress)

                        return try dec.decode(User.self, from: Data(bytesNoCopy: res_ptr.baseAddress!, count: size_ptr.baseAddress?.pointee ?? 0, deallocator: .none))
                    }
                }
            }
        }
    } catch {
        print("Error decoding JSON probably: \(error)")
        return User(name: "", age: 0)
    }
}

We can test that this is working by replacing ContentView with:

struct ContentView: View {
    var body: some View {
        VStack {
            let user = birthday(user: User(name: "Ellie", age: 24))
            Text("Post-birthday, \(user.name) is: \(user.age)!")
        }
        .padding()
    }
}

And you should see:

3 Metaprogramming at the boundaries

I want to give a quick preview of what is made possible by using compile-time code generation features (Template Haskell in Haskell, Swift Macros in Swift). This foreign function code generation API is exposed by the haskell-swift project, namely the swift-ffi Haskell library and haskell-ffi Swift package (since it is out of the scope of this tutorial, I will not cover how exactly the compile-time code-generation code works, but instead use the API provided by these libraries).

With these top-level foreign interaction facilities, coupled with the build tool also provided by haskell-swift, one can easily bootstrap and develop programs mixing Haskell and Swift.

Let us consider the same example where we define an idiomatic birthday :: User -> User function in Haskell and want to be able to call it from Swift as birthday(user: User) -> User

3.1 Haskell’s perspective

To expose the birthday function to Swift, we simply use the foreignExportSwift Template Haskell function. The whole module could look like this:

{-# LANGUAGE TemplateHaskell #-}
module MyLib where

-- ...
import Swift.FFI

data User
 = User { name :: String
        , age  :: Int
        }
        deriving stock    Generic
        deriving anyclass FromJSON
        deriving anyclass ToJSON

birthday :: User -> User
birthday User{age=x, name=y} = User{age=x+1, name=y}

$(foreignExportSwift 'birthday)

The key bit is the last foreignExportSwift call which will expose a C function with the marshalling-based calling convention we outlined above.

3.2 Swift’s perspective

On the Swift side, we want to use the dual @ForeignImportHaskell macro which generates a Swift function wrapper which in turn invokes the C function exposed by Haskell with the above marshalling strategy. The Swift file could look like:

import HaskellFFI

struct User: Codable {
    let name: String
    let age: Int
}

@ForeignImportHaskell
func birthday(cconv: HsCallJSON, user: User) -> User { stub() }

where birthday could be called e.g. as:

birthday(user: User(name: "Pierre", age: 55))

4 Remarks

The strategy of marshaling for foreign language boundary crossing is very robust and still performant, and is a great fit for the kind of mixed-language application we want to develop robustly.

Even though marshaling is required for robustly traversing the foreign language boundary, I will also explore, in a subsequent post, calling Haskell from Swift by instead coercing the memory representation of a Haskell value into a Swift one – this will mostly be a (very unsafe) and not-at-all robust curiosity, but it will give me an excuse to write a bit about low-level details in Haskell!

In yet another post, I also intend to introduce the hxs tool for bootstrapping Haskell x Swift projects and the libraries that make it so much easier to export Haskell functions and import them from Swift.

The haskell-x-swift-project-steps git repository has a commit matching the steps of this guide, so if anything is unclear you can just let the code speak by itself in checking the commits.

This project, blog post, and research regarding Swift interoperability with Haskell is being partially sponsored by Well-Typed, and is otherwise carried out in my own free time. If you’d also like to sponsor my work on Swift x Haskell interoperability with the goal of developing native macOS/iOS/etc applications, visit my GitHub sponsors page.

• 1 Records in Haskell
  • 1.1 Overloaded Record Dot
  • 1.2 Named Field Puns
• 2 Computed Properties
• 3 Conclusion

1 Records in Haskell

Haskell has so-called record types, which are also commonly known as structs, for instance, in C, Swift, and Rust. To define a square, one would write:

data Point
  = Point
    { x :: Int
    , y :: Int
    }

data Square
  = Square
    { topLeft     :: Point
    , bottomRight :: Point
    }

mySquare = Square{ topLeft = Point{x = 0, y = 0}
                 , bottomRight = Point{x = 2, y = 2} }
mySquareWidth = x (bottomRight mySquare) - x (topLeft mySquare)

In Haskell record types are just syntactic sugar for ordinary product types paired with functions that get and set these fields. In essence, the above is not fundamentally different from having the following standard product types and functions:

data Point = Point Int Int
data Square = Square Point Point

x, y :: Point -> Int
x (Point px _) = px
y (Point _ py) = py

topLeft, bottomRight :: Square -> Point
topLeft (Square tl _) = tl
bottomRight (Square _ br) = br

-- And setters...

1.1 Overloaded Record Dot

However, by turning on the OverloadedRecordDot syntax extension, you can use more syntactic sugar to project the fields of a record instead of using the field name as a standard function:

{-# LANGUAGE OverloadedRecordDot #-}
mySquareWidth = mySquare.bottomRight.x - mySquare.topLeft.x

which is neat! I like OverloadedRecordDot. It looks clean and feels more like using proper property of the record data type. It is also less ambiguous for an LSP to suggest the record properties of a data type by typing after the ., than it is to suggest functions to apply to the record type argument.

1.2 Named Field Puns

Since I’m already writing about records, I’ll mention another extension I quite enjoy: NamedFieldPuns.

Traditionally, when matching on a record, you can list the field names and bind variables to the value associated with that field. Continuing the above example:

area :: Square -> Int
area Square{topLeft = tl, bottomRight = br}
    = (br.x - tl.x) * (br.y - tl.y)

We know, however, that naming things is hard and best avoided. With NamedFieldPuns, instead of declaring the variable to be bound to the right of the field name, we have the field name be the variable bound to its value:

area :: Square -> Int
area Square{topLeft, bottomRight}
    = (bottomRight.x - topLeft.x) * (bottomRight.y - topLeft.y)

There are more record-related extensions, such as RecordWildCards or OverloadedRecordUpdate, which I will not get into, but that can also make life smoother when working with records.

2 Computed Properties

Computed properties are a (not-particularly-exclusive-to) Swift concept I’ve recently come accross while working on my interoperability between Haskell and Swift project.

Here is a definition from the Swift book section on (computed) properties:

Properties associate values with a particular class, structure, or enumeration. Stored properties store constant and variable values as part of an instance, whereas computed properties calculate (rather than store) a value. Computed properties are provided by classes, structures, and enumerations. Stored properties are provided only by classes and structures.

And a Swift example, where the volume is a property computed from the width, the height and the depth of a Cuboid:

struct Cuboid {
    var width = 0.0, height = 0.0, depth = 0.0
    var volume: Double {
        return width * height * depth
    }
}

C# also has a notion of computed properties. In Java, simple class methods computing a result from class properties can also be seen as some sort of computed property, or, really, class methods in any object oriented language.

In Haskell, as basically everything else, you can think of computed properties as… just functions. But there is one key element to computed properties that makes them different from just functions – them being called using dot syntax at a value of a record type just like any other property, and the evoking the idea of describing a property of the record type.

Can we have that kind of computed properties in Haskell? Well, of course!

Following the examples in previous sections, consider the area function to be conceptually a computed property of a rectangle, as it uses the two Square properties (topLeft and bottomRight) to compute a new one. Ultimately, we want to be able to type:

> mySquare.area
6

To achieve this, we need to emulate the behaviour of field accessors. The key insight is to use the HasField class just like default field accessors do. HasField enables so-called record field selector polymorphism and allows us to not only define functions to operate on any record type with eg a field named name, it allows us to define field accessors for non-record types, and, ultimately, allows us to create computed properties. The ability to define our own instances of HasField is also documented in the user guide under virtual record fields.

To make the area function a field accessor, thereby making it a record-dot-enabled-computed-property, we instance HasField using the field name area (which is a defined as a type-level string in the first argument to HasField):

{-# LANGUAGE GHC2021, OverloadedRecordDot, DataKinds #-}
import GHC.Records

instance HasField "area" Square Int where
    getField = area

You can now write some_square.area to compute the area of the square based on its record properties.

Here’s an example of a full program defining another computed property and printing it:

{-# LANGUAGE GHC2021, OverloadedRecordDot, DataKinds #-}

import GHC.Records

data User = User
  { birthYear :: Int
  , name :: String
  }

instance HasField "age" User Int where getField = age

age :: User -> Int
age u = 2023 - u.birthYear

user :: User
user = User{birthYear=1995, name="Robert"}

main = print user.age

3 Conclusion

This has been a whirlwhind thought kind of post. I am not currently using this in any project. I thought “I suppose we can also have this neat properties sugar” and tried it, this is only my exposition of the idea.

In my opinion, this could be handy as some functions can really be better thought of as properties of a datatype, and doing so doesn’t preclude you from also using it as a function in cases where it reads more naturally (and of course, pass it on to higher order functions). LSP based autocompletion of (computed) properties after the dot might be another positive factor. It is probably also a Haskell anti-pattern for some!

I’m left wondering: is anyone out there doing this?

• 1 Hello, Swift, it’s Haskell!
  • 1.1 Setting up the SwiftUI app
  • 1.2 Setting up a Haskell foreign library
  • 1.3 Linking the Haskell library with the executable
  • 1.4 The RTS must be initialized
• 2 Remarks
  • 2.1 Further Reading

This is the first part of an in-depth guide into developing a native applications for Apple platforms (macOS, iOS, etc.) using Haskell with Swift and SwiftUI. This is the first in a series of blog posts – covering the set-up required to call Haskell functions from Swift in an XCode project using SwiftUI. In future installements of the series, I intend to at least discuss calling functions with idiomatic Haskell types with Swift ones (both with and without marshaling), SwiftUI observation, and iOS development which requires GHC to produce code for the iOS compilation target.

At the time of writing I’m using XCode 15, Cabal 3.10, and GHC 9.8. There will be some features I use that are only available in these recent versions, however, the general idea of interoperability between Haskell and Swift stands on its own regardless – the now 7 year old swift-haskell-tutorial is still similarly relevant and greatly informed my approach, despite the end result being considerably different.

The end goal is to create a multi-(apple)-platform application whose UI is programmed in Swift using SwiftUI while the data and logic of the application is implemented in Haskell which is called from Swift.

The series of blog posts is further accompanied by a github repository where each commit matches a step of this tutorial. If in doubt regarding any step, simply checking the matching commit for absolute confidence you are understanding the practical step correctly. Visit this link to the haskell-x-swift-project-steps repository! Furthermore, I’m writing a build tool that will facilitate setting up and building a project like this without having to go through all the manual steps: haskell-swift.

This write-up has been cross-posted to Well-Typed’s Blog.

1 Hello, Swift, it’s Haskell!

In this part we are only concerned with getting our Hello, World! going.

1. We’ll setup a Haskell (foreign) library exporting a function hs_factorial that returns the factorial of integer, using the C FFI
2. Setup a SwiftUI app that calls hs_factorial
3. Compile the Haskell code into a shared library
4. Create a Swift module HaskellFramework to export the Haskell functions (imported from the stub C header files), and setup linking against the Haskell shared library.
5. Import HaskellFramework into the SwiftUI app to be able to successfully call hs_factorial and display the result on the screen of the running application.

The following diagram (rendered by Diagon) describes how the Swift executable and Haskell libraries are going to be connected from a not-too-far-away perspective. It might be useful to consult this diagram once in a while throughout the post!

┌───────────────┐┌───────────┐┌───────────────────────┐┌───────────┐
│Haskell library││cbits      ││gen-dynamic-settings.sh││RTS headers│
└┬──────────────┘└┬─────────┬┘└─────────────┬─────────┘└┬──────────┘
┌▽────────────────▽───────┐┌▽──────────────┐│           │
│Haskell foreign library  ││Headers (cbits)││           │
└┬───────────────────────┬┘└─────────────┬─┘│           │
┌▽─────────────────────┐┌▽──────────────┐│  │           │
│Shared dynamic library││Headers (stubs)││  │           │
└┬────────────────────┬┘└┬──────────────┘│  │           │
 │            ┌───────│──│───────────────┘  │           │
 │            │┌──────│──┘                  │ ┌─────────┘
 │┌───────────▽▽┐┌────▽─────────────────────▽─▽┐
 ││Clang modules││DynamicBuildSettings.xcconfig│
 │└┬────────────┘└┬────────────────────────────┘
 │ │┌─────────────▽────────┐
 │ ││BuildSettings.xcconfig│
 │ │└┬─────────────────────┘
┌▽─▽─▽──────┐
│SwiftUI App│
└───────────┘

1.1 Setting up the SwiftUI app

Let’s set-up a simple XCode project using SwiftUI for the main interface. Fire up XCode and create a macOS Application, named SwiftHaskell, using SwiftUI, excluding tests. Choose a Personal Team rather than None - you might have to create a (free of charge) one.

In the newly-created project there should exist two files: SwiftHaskellApp.swift and ContentView.swift. We can change right away ContentView.swift to display the result of calling hs_factorial(5), even though hs_factorial is not yet in scope:

import SwiftUI

struct ContentView: View {
    var body: some View {
        VStack {
            Text("Hello, Haskell: \(hs_factorial(5))!")
        }
        .padding()
    }
}

Before proceeding to the Haskell side, create a New File > Configuration Settings File (also known as a .xcconfig file) named BuildSettings.xcconfig. We’ll use this file to write all our build settings textually instead of using XCode’s build settings navigator.

To use our .xcconfig file for the project settings, under Info > Configurations in the project tab, select the BuildSettings file. For the configuration to show up in XCode, the .xcconfig must be in the tree navigator (which happens by default if you created the module within XCode). You can read more, or see exactly how to set an .xcconfig file as the configuration, in this write-up on xcconfig by NSHipster.

Even though we are setting the .xcconfig file manually (and also e.g. initializing the XCode project), it is possible to resort to an exclusively programatic approach using so-called XCode project generators such as XCodeGen and Tuist.

1.2 Setting up a Haskell foreign library

Create a folder haskell-framework within the XCode project, cd into it, and follow from there.

We’re jumping straight into a full-fledged Haskell projected managed with cabal, where we define a shared library using the foreign-library stanza.

Start with a normal cabal file with a library stanza that exposes MyLib (by running cabal init within haskell-framework), and add the function hs_factorial to MyLib that operates on CInts:

module MyLib where
import Foreign.C

hs_factorial :: CInt -> CInt
hs_factorial x = product [1..x]

The organization of the code here isn’t terribly important. Perhaps in a real project you could want to, for instance, only use C types like CInt in the foreign library bits.

In the cabal file, add a foreign-library stanza with

foreign-library haskell-foreign-framework
    type: native-shared

    -- This should work on Mac, despite being undefined behaviour
    -- See https://www.hobson.space/posts/haskell-foreign-library/ (great read)
    options: standalone

    -- We copy the C stub headers to a folder in the root.
    -- If you have foreign-export declarations in the library
    -- be sure to add this flag there too (so all stubs get added
    -- to the `haskell-framework-include` folder)
    ghc-options: -stubdir=haskell-framework-include

    other-modules: MyForeignLib
    build-depends: base, haskell-framework
    hs-source-dirs: flib

Unfortunately, options: standalone is only officially supported (and required) by Windows, even though it is exactly what we need. However, unofficially, a macOS distribution should be able to safely use this option – for more information see this write-up on foreign libraries explaining why this option is undefined for macOS. In the future, this might work out of the box without being undefined behaviour, or the behaviour on macOS may have changed s.t. this no longer works… but let’s hope for the former. Additionally, we pass -stubdir for GHC to output the C stub header files to a directory haskell-framework-include. Do add this automatically generated directory to .gitignore.

Create the file flib/MyForeignLib.hs that declares a foreign export of hs_factorial imported from MyLib and foreign exports it:

{-# LANGUAGE ForeignFunctionInterface #-}
module MyForeignLib where
import Foreign.C
import MyLib (hs_factorial)
foreign export ccall hs_factorial :: CInt -> CInt

It doesn’t seem that re-exporting the function from MyLib when it is foreign exported from there is enough for it to be included in the shared library (might be a bug), we do need the foreign export here rather than in MyLib.

Running cabal build should now generate a haskell-framework-include folder with a MyForeignLib_stub.h, and a libhaskell-foreign-framework.dylib shared library somewhere under dist-newstyle (you can find . -name libhaskell-foreign-framework.dylib to find it)

We’ll test C program against this library to check whether it works as expected. Create scripts/test-haskell-foreign-lib.sh with a script that compiles a main function in C which calls hs_factorial. A few notes:

• We need to pass the path to the built shared library ($HS_FLIB_PATH) to the compiler.
• We need to pass the path to the headers ($HS_HEADERS_PATH).
• We hardcode into the executable the path to the shared library as an rpath search path (just for testing purposes). When building the macOS app, XCode will add @executable_path/../Frameworks to the rpath search path, so we can simply copy the shared library the Apple-blessed location (Frameworks).
• We need to call hs_init and hs_exit to init the runtime system (see the relevant GHC user guide section).
• We need to compile the C library using ghc, as it will automatically include and link the rts headers and library. To use a C compiler we’d also need to find the rts headers and library of our Haskell installation.

#!/usr/bin/env bash

set -e

if ! test -f "haskell-framework.cabal"; then
    echo "Run this script from the root of your project!"
    exit 1
fi

HS_FLIB_PATH=$(dirname $(find . -name libhaskell-foreign-framework.dylib))
HS_HEADERS_PATH=haskell-framework-include

echo "
#include <stdio.h>
#include <MyForeignLib_stub.h>
#include <HsFFI.h>
int main(void) {
    hs_init(NULL, NULL);
    printf(\"%d\n\", hs_factorial(5));
    hs_exit();
    return 0;
}
" > conftestmain.c

# We use `ghc` instead of `gcc` because otherwise we also need to provide the
# include and lib path of the runtime system (Rts)
ghc -no-hs-main -o conftest conftestmain.c \
    -lhaskell-foreign-framework \
    -I"$HS_HEADERS_PATH" \
    -L"$HS_FLIB_PATH" \
    -optl-Wl,-rpath,"$HS_FLIB_PATH"

RESULT=$(./conftest)

if [ 120 -eq $RESULT ]; then
    echo "Foreign library successfully called!"
else
    echo "Bad bad foreign library!"
    exit 1
fi

rm -f conftest*

You should get Foreign library successfully called! when this script is run.

1.3 Linking the Haskell library with the executable

Our recipe for invoking a foreign exported Haskell function in Swift:

1. Create a Swift module exporting Haskell functions through a module map pointing to the headers exporting the Haskell functions.
2. Extend the module search path with the location of your new module map.
3. Import that module as a module in the SwiftUI code, and use the desired function.
4. At link time, the shared library with the symbols used by the program must be linked against, and must be found in the run-path which can be done by copying the shared library into the app bundled Frameworks folder.

We create a module map file listing all the headers exporting Haskell functions to define Swift modules where Haskell functions will live, using Clang’s module system. A module map looks something like

module HaskellFramework {
    header "haskell-framework/haskell-framework-include/MyForeignLib.h"
    export *
}

and can be imported into Swift code with import HaskellFramework, as long as the module map is available as module.modulemap in the import search path. As one might expect, importing this module brings into scope all names exported from the listed header(s).

Specifically, we will use the inferred submodules feature of modules to create our module map. With inferred submodules, we can simply define an umbrella directory with headers and get a submodule for each header in that directory (arbitrarily nested, where a header A/B/C.h becomes a submodule named MainModule.A.B.C)

In the root of the XCode project, write a module.modulemap file:

module HaskellFramework {
    umbrella "haskell-framework/haskell-framework-include"
    
    explicit module * {
        export *
    }
    
}

The umbrella keyword specifies the directory where to find the header files for our submodules, and the explicit module * lines are the inferred submodule part, as each header will result in a declaration roughly like explicit module HeaderName { header "umbrella/HeaderName.h" ... }. In effect, our module map above will expand to:

module HaskellFramework {
    explicit module MyForeignLib {
        header "haskell-framework/haskell-framework-include/MyForeignLib.h"
        export *
    }
}

Again, to be clear, this is what our original module.modulemap using the umbrella keyword currently expands to, not the file we wrote.

Having written our module.modulemap, we need to extend the compiler’s import search path to find this module map. As we’ve already set-up our xcconfig-based configuration, this amounts to writing into BuildSettings.xcconfig:

SWIFT_INCLUDE_PATHS=$(PROJECT_DIR)

This is equivalent to changing the Swift Compiler - Search Paths > Import Paths build setting in XCode (in fact, by inspecting that setting on the rightmost inspector panel, you will find the corresponding xcconfig name is indeed SWIFT_INCLUDE_PATHS – this is also all explained in the xcconfig article).

Returning to ContentView.swift, where hs_factorial is being called, you should be able to add at the top of the file, and have XCode successfully recognize:

import HaskellFramework.MyForeignLib_stub

Even though the import is recognized, it will not compile successfully. The reason is our stub header (MyForeignLib_stub.h) includes <HsFFI.h> which cannot be found by XCode. We need to extend our Header Search Path with the path to the RTS headers.

Currently, our BuildSettings.xcconfig can only contain statically known information. Fortunately, we can #include other xcconfig files (that may have been generated dynamically) in our BuildSettings.xcconfig (as described by the xcconfig write-up). We do this by adding the following include directive in the BuildSettings.xcconfig file:

#include "DynamicBuildSettings.xcconfig"

We will generate DynamicBuildSettings.xcconfig with a script haskell-framework/scripts/gen-dynamic-settings.sh that calls the

ghc-pkg field rts include-dirs --simple-output

to figure out the rts include path of the existing GHC installation.

We extend HEADER_SEARCH_PATHS, the xcconfig variable listing the paths where XCode will search for headers when building, with the path to the RTS header files:

#!/usr/bin/env bash

set -e
if ! test -f "haskell-framework/haskell-framework.cabal"; then
    echo "Run this script from the root of your XCode project!"
    exit 1
fi

echo "
HEADER_SEARCH_PATHS=\$(inherit) $(ghc-pkg field rts include-dirs --simple-output | tr ' ' '\n' | tail -n1)
" > DynamicBuildSettings.xcconfig

echo "Created DynamicBuildSettings.xcconfig!"

Do add DynamicBuildSettings.xcconfig to .gitignore. The literal string $(inherit) is xcconfig syntax for inheriting the options set before applying this configuration. Furthermore, asking for the include-dirs of rts outputs two directories:

/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/ffi
/Users/romes/.ghcup/ghc/9.8.1/lib/ghc-9.8.1/lib/../lib/aarch64-osx-ghc-9.8.1/rts-1.0.2/include

However, the ffi header is already included in a module by default in XCode applications, so we need to cut it out of the search paths to avoid a Redefinition of module 'FFI' error (tr combined with tail -n1 select just the path to the RTS headers).

The function is now found by XCode as the module it is defined in compiles successfully and brings the function into scope. However, building the program will fail with a link time error: even though we instructed the compiler to find the definitions of the Haskell functions we want to use and the module they are exported from, we have not linked against the library where the actual symbols are defined.

The Haskell foreign library created in a previous section compiles to a shared dynamic library. To link against it when building our Swift application we need to pass -lhaskell-foreign-framework to the compilation toolchain and instruct it on where to find this library. The first step can be done in two compatible (as in both can co-exist) ways:

• Add a link "haskell-foreign-framework" declaration to the module map (explained here)
  • There is a note about this feature not yet being widely supported in the reference page, however, it is works to link the library in my machine with XCode 15.
• Add the -lhaskell-foreign-framework flag to the OTHER_LDFLAGS build setting in BuildSettings.xcconfig. You can do this even if you’ve also used the link directive.

After adding the link declaration, your module.modulemap should contain:

module HaskellFramework {
    umbrella "haskell-framework/haskell-framework-include"
    
    explicit module * {
        export *
    }

    link "haskell-foreign-framework"
}

Secondly, we need to add the shared library path to the library search path and make it available at runtime by copying it to the Frameworks folder that is bundled with the application. By copying the library to this folder we ensure it can be found when dynamically loaded at runtime: the library install name is relative to @rpath, i.e. it is a run-path dependent library, and the run-path dependencies of XCode built executables are searched for in the Frameworks folder, relatively to the executable path (@executable_path/../Frameworks).

A run-path dependent library is a dependent library whose complete install name is not known when the library is created (see How Dynamic Libraries Are Used). Instead, the library specifies that the dynamic loader must resolve the library’s install name when it loads the executable that depends on the library.

To use run-path dependent libraries, an executable provides a list of run-path search paths, which the dynamic loader traverses at load time to find the libraries.

In practice, we achieve this by extending the LIBRARY_SEARCH_PATHS setting dynamically and add a “Copy” Build Phase which copies the shared library to the listed Frameworks folder. At this time, I do not know how to do this Copy outside of XCode – do shoot me a text if you know how. It is also unfortunate that we have to hardcode the path to the dynamic library there, instead of computing it at build time.

Find the path to the foreign library by running, in the haskell-framework directory:

cabal list-bin haskell-foreign-framework

Then, under the Build Phases tab of the project settings, add (by clicking in the little plus sign) a New Copy Files Phase. Then, clicking in the plus sign of the new listing of files to copy, add the haskell-foreign-framework .dylib (the shared library) that lives at the path found by running the above command by clicking on “Add Other”.

To the haskell-framework/scripts/gen-dynamic-settings.sh, add the following lines before echoing to the file

pushd . > /dev/null
cd haskell-framework
FLIB_PATH=$(cabal list-bin haskell-foreign-framework)
popd > /dev/null

and to what is written to DynamicBuildSettings.xcconfig add the following line

LIBRARY_SEARCH_PATHS=\$(inherit) $(dirname $FLIB_PATH)

At this point, after regenerating the dynamic build settings, you should be able to link the application successfully, and run it.

1.4 The RTS must be initialized

Surprise! Running the application will fail at runtime, when hs_factorial is called. To call Haskell functions from an executable written in another language, one must first initialize the GHC runtime system, and terminate it when appropriate. We need to call the functions hs_init and hs_end, exposed in HsFFI.h. We will write two wrapper functions in our foreign library to invoke instead, as suggested in the FFI chapter of the GHC user guide.

We create a cbits folder in the haskell-framework Haskell project to put our C files and headers, and add them to the foreign-library stanza of the cabal file:

include-dirs: cbits
c-sources: cbits/MyForeignLibRts.c
install-includes: MyForeignLibRts.h

You can see what these options do in this cabal user guide section. We create cbits/MyForeignLibRts.c wrapping the calls to hs_init and hs_end as described in the FFI chapter linked above:

#include <stdlib.h>
#include <stdio.h>
#include <HsFFI.h>

HsBool flib_init() {

    printf("Initialising flib\n");

    // Initialise Haskell runtime
    hs_init(NULL, NULL);

    // Do other library initialisations here

    return HS_BOOL_TRUE;
}

void flib_end() {
    printf("Terminating flib\n");
    hs_exit();
}

It might seem that you could foreign import these functions into the Haskell library and re-export them with foreign export, however, if they are exported from Haskell, they themselves require the RTS to be initialised, effectively defeating the purpose of being functions that initialise the RTS. Therefore, we write a header file that we ship with the library for it to be included by the Swift project. The file cbits/MyForeignLibRts.h contains:

#include <HsFFI.h>

HsBool flib_init();
void flib_end();

Back to the Swift side, we need to augment our module map with a module mapping to the RTS initialisation wrapper header. We add a second submodule declaration:

explicit module RTSManage {
   header "haskell-framework/cbits/MyForeignLibRts.h"
}

The cbits/MyForeignLibRts.c symbols will be included in the shared dynamic library.

You can re-buid the haskell library and re-generate the dynamic settings with a script ./build-haskell in the root of the XCode project:

#!/usr/bin/env bash

set -e

if ! test -d "SwiftHaskell.xcodeproj"; then
    echo "Run this from the SwiftHaskell XCode project root!"
    exit 1
fi

pushd . >/dev/null
cd haskell-framework/
cabal build all --allow-newer
./scripts/test-haskell-foreign-lib.sh
popd >/dev/null
./haskell-framework/scripts/gen-dynamic-settings.sh

echo "Done."

Finally, in SwiftHaskellApp.swift, we extend the @main App by overriding the init() function: calling flib_init() to initialise the runtime system and setting up an observer to call flib_end() to end the runtime system when the application terminates. We need only import HaskellFramework.RTSManage to bring these functions into scope:

@main
struct SwiftHaskellApp: App {

    init() {
        flib_init()

        NotificationCenter.default.addObserver(forName: NSApplication.willTerminateNotification, object: nil, queue: .main) { _ in
            // terminating
            flib_end()
        }
    }

    ...
}

Running your application should work and proudly print 120 on the screen.

2 Remarks

We’ve come to the end of the first installment in this blogpost series. Next up is communicating more interesting data types (both with and without marshalling), making things more ergonomic to use, SwiftUI observation, iOS compilation, and perhaps developing a simple model app.

The haskell-x-swift-project-steps git repository has a commit matching each of the steps of this guide, so if anything is unclear you can just let the code speak by itself in checking the commits.

This project, blog post, and research regarding Swift interoperability with Haskell is being partially sponsored by Well-Typed, and is otherwise carried out in my own free time. If you’d also like to sponsor my work on Swift x Haskell interoperability with the goal of developing native macOS/iOS/etc applications, visit my GitHub sponsors page.

2.1 Further Reading

• swift-haskell-tutorial by nanotech
• Haskell foreign library and options: standalone
• Using the FFI with GHC
• xcconfig by NSHipster
• Clang module
• Run-path dependent libraries
• Cabal user guide

• 1 Compiler Toolchains
• 2 The runtime-retargetable future of GHC
• 3 Introducing ghc-toolchain
• 4 Migration to ghc-toolchain
• 5 Future work
• 6 Conclusion

GHC, like most high-level language compilers, depends upon a set of tools like assemblers, linkers, and archivers for the production of machine code. Collectively these tools are known as a toolchain and capture a great deal of platform-dependent knowledge.

Traditionally, developers generate a ./configure script using the venerable autoconf tool, then users execute this script when they install a GHC binary distribution. The ./configure script determines the location of programs (such as the C compiler) and which options GHC will need to pass them.

While this autoconf-centric model of toolchain configuration has served GHC well, it has two key issues:

• For cross-compiling to a different platform, it would be highly valuable to users if GHC would become a runtime-retargetable compiler (like rustc and go). That is, the user should be able to download a single GHC binary distribution and use it to compile not only for their local machine, but also any other targets that GHC supports.

• The ten-thousand-line sh file that is GHC’s ./configure script has historically been challenging to maintain and test. Modifications to the ./configure script are among the most risky changes routinely made to the compiler, because it is easy to introduce a bug on some specific toolchain configuration, and infeasible to test all possible configurations in CI.

To address these issues, we are introducing ghc-toolchain, a new way to configure the toolchain for GHC, which will eventually replace the existing toolchain configuration logic in the ./configure script. Its main goal is to allow new compilation toolchains to be configured for GHC at any point in time, notably after the compiler has been installed. For example, calling ghc-toolchain --triple=x86_64-w64-mingw32 will configure a compilation toolchain on the host machine capable of producing code for an x86_64 machine running Windows using MinGW. This is an important step towards making GHC runtime-retargetable, and since ghc-toolchain is implemented in Haskell, it will be much easier to modify and test than the ./configure script.

In this post we explain in more detail how GHC interacts with the system toolchain and how ghc-toolchain facilitates our future goal of making GHC a runtime-retargetable compiler.

This post was a result from my internship with the GHC team at Well-Typed this summer. It was originally made available in the Well-Typed Blog. I’m very grateful to Matthew, Ben, Sam, and Andreas for mentoring me throughout my internship and reviewing previous iteration of this post. It wouldn’t have been possible to complete this project without Ben Gamari’s help – Ben wrote the first draft of ghc-toolchain and guided me through many weird but interesting toolchain quirks and toolchain configuration bugs while developing ghc-toolchain to its full mergeable extent in GHC.

1 Compiler Toolchains

GHC cannot produce executables from Haskell programs in isolation – it requires a correctly configured toolchain to which it can delegate some responsibilities. For example, GHC’s native code generator backend is capable of generating assembly code from a Haskell program, however, producing object code from that assembly, and linking the objects into an executable, are all tasks done by the compilation toolchain, which is invoked by GHC using the flags that were configured for it.

Configuring a compiler toolchain is about locating the set of tools required for compilation, the base set of flags required to invoke each tool, and properties of these tools. For example, this might include:

• determining various characteristics of the platform (e.g. the word size).
• probing to find which tools are available (the C compiler, linker, archiver, object merging tool, etc.),
• identifying which flags GHC should pass to each of these tools,
• determining whether the tools support response files to work around command-line length limits, and
• checking for and working around bugs in the toolchain.

At the moment, when a GHC binary distribution is installed, the ./configure script will perform the above steps and store the results in a settings file. GHC will then read this file so it can correctly invoke the toolchain programs when compiling Haskell executables.

To cross-compile a Haskell program, a user must build GHC from source as a cross-compiler (see the GHC wiki). This requires configuring a cross-compilation toolchain, that is, a toolchain that runs on the machine compiling the Haskell program but that produces executables to run on a different system. It is currently a rather involved process.

2 The runtime-retargetable future of GHC

A key long-term goal of this work is to allow GHC to become runtime-retargetable. This means being able to call ghc --target=aarch64-apple-darwin and have GHC output code for an AArch64 machine, or call ghc --target=javascript-ghcjs to generate Javascript code, regardless of the platform ghc is being invoked on.

Crucially, this requires the configuration step to be repeated at a later point, rather than only when the GHC binary distribution is installed. Once GHC is fully runtime-retargetable, this will allow you to use multiple different toolchains, potentially targeting different platforms, with the same installed compiler.

• At the simplest level, you might just have two different toolchains for your host platform (for example, a gcc-based toolchain and a clang-based toolchain), or you might just configure a toolchain which uses the new mold linker rather than ld.gold.

• In a more complex scenario, you may have a normal compiler toolchain as well as several different cross-compiler toolchains. For example, a toolchain which produces Javascript, a toolchain which produces WebAssembly, a toolchain which produces AArch64 object code and so on.

The idea is that the brand new ghc-toolchain will be called once to configure the toolchain that GHC will use when compiling for a target, then ghc --target=<triple> can be called as many times as needed. For example, if you have an x86 Linux machine and wish to produce code for AArch64 devices, the workflow could look something like:

# Configure the aarch64-apple-darwin target first
# (We only need to do this once!)
ghc-toolchain --triple=aarch64-apple-darwin --cc-opt="-I/some/include/dir" --cc-linker-opt="-L/some/library/dir"

# Now we can target aarch64-apple-darwin (as many times as we'd like!)
ghc --target=aarch64-apple-darwin -o MyAwesomeTool MyAwesomeTool.hs
ghc --target=aarch64-apple-darwin -o CoolProgram CoolProgram.hs

3 Introducing ghc-toolchain

ghc-toolchain is a standalone tool for configuring a toolchain. It receives as input a target triplet (e.g. x86_64-deb10-linux) and user options, discovers the configuration, and outputs a “target description” (.target file) containing the configured toolchain.

At the moment, .target files generated by ghc-toolchain can be used by GHC’s build system (Hadrian) by invoking ./configure with --enable-ghc-toolchain. Otherwise, Hadrian reads the configuration from a .target file generated by ./configure itself.

In the future, ghc-toolchain will be shipped in binary distributions to allow new toolchains to be added after the compiler is installed (generating new .target files). GHC will then be able to choose the .target file for the particular target requested by the user.

From a developer standpoint, ghc-toolchain being written in Haskell makes it easier to modify in future, especially when compared to the notoriously difficult to write and debug ./configure scripts.

4 Migration to ghc-toolchain

We are migrating to ghc-toolchain in a staged manner, since toolchain configuration logic is amongst the most sensitive things to change in the compiler. We want to ensure that the configuration logic in ghc-toolchain is correct and agrees with the logic in ./configure. Therefore, in GHC 9.10 ghc-toolchain will be shipped and validated but not enabled by default.

To validate ghc-toolchain, GHC will generate .target files with both ./configure and ghc-toolchain and compare the outputs against each other, emitting a warning if they differ. This means we will be able to catch mistakes in ghc-toolchain (and in ./configure too!) before we make ghc-toolchain the default method for configuring toolchains in a subsequent release. This mechanism has already identified plenty of issues to resolve.

5 Future work

Despite ghc-toolchain bringing us closer to a runtime-retargetable GHC, there is still much work left to be done (see #11470). The next step is to instruct GHC to choose between multiple available .target files at runtime, instead of reading the usual settings file (tracked in #23682).

Beyond that, however, there are many open questions still to resolve:

• How will the runtime system, and core libraries such as base, be provided for the multiple selected targets?
• How will this fit into ghcup’s installation story?
• How will cabal handle multiple targets?

At the moment, binary distributions include the RTS/libraries already compiled for a single target. Instead, we are likely to need some mechanism for users to recompile the RTS/libraries when they configure a new target, or to download ready-built versions from upstream.

Moreover, accommodating TemplateHaskell under runtime retargetability is particularly nontrivial, and needs more design work.

6 Conclusion

ghc-toolchain is a new tool for configuring toolchains and targets. It improves on GHC’s existing ./configure-based configuration workflow by allowing multiple targets’ toolchains to be configured at any time, and by making maintenance and future updates to the toolchain configuration logic much easier. However, toolchain configuration is a challenging part of the compiler, so we’re being conservative in migrating to ghc-toolchain, and carefully validating it before making it the default.

Moreover, ghc-toolchain is an important step towards making a runtime-retargetable GHC a reality, though there is still much work left to do. We are grateful to all the GHC developers involved in working towards runtime-retargetability.

Well-Typed is able to work on GHC, HLS, Cabal and other core Haskell infrastructure thanks to funding from various sponsors. If your company might be able to contribute to this work, sponsor maintenance efforts, or fund the implementation of other features, please read about how you can help or get in touch.

• 1 Haskell’s Unicode Syntax Extension
• 2 Digraphs in Vim
• 3 Conclusion

1 Haskell’s Unicode Syntax Extension

Haskell (well, GHC Haskell) features an extension called UnicodeSyntax. When enabled, this extension allows the use of certain unicode symbols in place of their corresponding keywords. A great example is the forall keyword being equivalent to the unicode symbol ∀, the two of which can be used interchangebly when UnicodeSyntax is enabled.

Furthermore, with Haskell being a unicode-friendly language, one can define common Haskell functions, operators or type variables using unicode symbols – which doesn’t even require UnicodeSyntax to be enabled. For example, one can define the predicate ∈ on lists as an alias for elem as follows:

-- 5 ∈ [1,3,5] == True
(∈) :: ∀ α. Eq α => α -> [α] -> Bool
(∈) = elem

In practice, I use just a handful of unicode symbols both as keywords and as identifiers, but a mostly comprehensive list of the keywords that have unicode alternatives is presented in the GHC user’s guide UnicodeSyntax extension page. Specifically, in most of my programs you can be sure to find the following:

• ∀ instead of forall, which is faster to input than the whole word.
• A lot of unicode type variables, α, β, τ, σ, δ, κ, ρ – they are really easy to type too.
• ⊸ instead of %1 ->, to use the so-called “lollipop” notation for linear functions.

In my opinion, those are low-hanging niceties (with vim) that make the program look better overall, but there are others that I haven’t yet reached for which you may still find good/useful. For example, there’s a library in hackage, containers-unicode-symbols, which exposes multiple unicode variants of functions on containers (Maps,Sets,…) such as ∈,∉,∅,∪,∩,⊆,⊈.

Finally, I usually add default-extensions: UnicodeSyntax to my cabal file to make the extension available by default on all modules. However, you can also enable it on a per module basis as usual with {-# LANGUAGE UnicodeSyntax #-} at the top of the module.

2 Digraphs in Vim

To experiment with UnicodeSyntax, or if you’re already convinced that using unicode symbols makes the programs nicer to look at, all that’s left is to be able to input these symbols easily. Vim has a built-in feature called digraphs that makes inputting unicode symbols a joy. The feature is called digraphs because it maps combinations of exactly two letters to a unicode symbol (see also :help digraph).

To input a digraph, in insert mode, press Ctrl+k followed by the two letters which define the digraph. Here are a few useful, built-in, combinations:

• Ctrl-k+FA inputs ∀.
• Ctrl-k+
  • a* inputs α.
  • b* inputs β.
  • t* inputs τ.
  • In general, Ctrl-k+letter+* inputs the greek letter variant of that letter
• Ctrl-k+(- inputs ∈.
• Ctrl-k+:: inputs ∷.
• Ctrl-k+=> inputs ⇒.
• Ctrl-k+-> inputs →.
• Ctrl-k+TE inputs ∃.

Besides the built-in ones, it’s very useful to define your own digraphs. Both for customization/personalization and ergonomics, but also to introduce digraphs which simply do not exist by default.

To create a digraph, use the digraph VimScript keyword with the two characters that input it, and the decimal numeric representation of the Unicode character it is mapped to. In the following .vimrc snippet, I defined the digraph ll with 8888 (the unicode decimal representation of ⊸), which effectively maps Ctrl-k+ll to ⊸.

digraph ll 8888 " ⊸

3 Conclusion

Concluding, vim makes it really easy, through digraphs, to input unicode symbols which are understood by Haskell, and even more so with its UnicodeSyntax extension. Combining these features we can easily write more beautiful Haskell programs. I’d argue it’s as fast to write

lid :: ∀ α. α ⊸ α

as it is to write

lid :: forall a. a %1 -> a

while the former is arguably more aesthetically pleasing.

• 1 Ghengin
  • 1.1 Bullets on Technical Details
  • 1.2 The Small Victories
  • 1.3 A peek into the code

1 Ghengin

I’ve been working the past month or two in a game engine titled Ghengin (pronounced /ɡɛn-ʤɪn/, never /ɡɛn-ɡɪn/). This is not yet a release, and version 0.1.0 is far into the future. However, I’ve come a long way and I’d like to share a few pictures of my progress. This post was migrated from the discussion at the Haskell Discourse

The demo I’ve been working on is based on Sebastian Lague’s series Procedural Planets. It is a showcase of procedurally generated planets you can move around in and tweak the procedural generation parameters of the planets to create oceans and continents.

1.1 Bullets on Technical Details

I hope to, soon enough, write a more substantial explanation of the engine’s technical challenges and overall design decisions so far, and on the game developer’s facing side of the engine. In the meantime, here are a few key points regarding the technical feats of the engine along with the main libraries it currently depends on, which help create a picture of how it is working:

• The renderer is written using the great bindings to the Vulkan API

• The shaders are crucial in the overall design, and a lot of code depends on their definition (e.g. preparing render pipelines, allocating descriptor sets and textures, everything materials related …). The shaders are written using FIR, an amazing shader language embedded in Haskell!

• The entity management, scene graph and render queue are done/created through the apecs entity component system.

• Vectors and matrices are from geomancy

• GLFW-b for window management and user input (used as the window backend for vulkan)

• The dear-imgui bindings for the GUI

• JuicyPixels for loading textures

FIR is a really cool shader library and unlike any you’ve likely tried before (it’s embeded in Haskell, but that’s just the start). The shader’s “interfaces” are defined at the type level, and in ghengin that type information is used to validate the game-developer-defined-materials. In short, if you define materials incompatible with your shaders, the program will fail at compile time

1.2 The Small Victories

To give a general sense of progress, I put together a small roadmap of victories attained while developing the engine, both in words and in screenshots.

• The very first achievement was rendering a triangle (Fig. 2). This is a given classic in graphics programming.

• Then, I rendered a simple cube and was able to rotate it with a simple model transform matrix (Fig. 3).

• Later, I got a perspective camera which could move around the world. I was generating spheres at this point and the colors show that I was getting closer to generating the normals right too (Fig. 4).

• I managed to integrate dear-imgui into the renderer after that, and even fixed upstream a dreaded off by one error which kept making the GUI behave funny and crash. I was also experimenting with simple diffuse lighting here (Fig. 5).

• With the GUI in place, I started focusing on developing planets by generating single sphere and modifying the height value of each point on the sphere by noise value: generating terrain and mountains (Fig. 6).

• After the terrain generation I spent some long weeks on the internals of the renderer before achieving more visual results with the exception of the following color-based-on-height-relative-to-min-and-max-heights planet (Fig. 7). Those weeks were spent in internal technical challenges which I hope to describe on a subsequent post with the resulting design and implementation (and hopefully avoid to some extent the arduous process of understanding and reaching a design and implementation).

• This week, with the material system working great for a first iteration, I spent finally some more time on the procedural planets: I added specular highlights to the lighting model (using the blinn-phong model) and added a (gradient based) texture to the planet that is sampled according to the height of each point in the planet. The result is a nicely lit planet with colors depending on the height: lower height -> blue for water, middle -> green for grass, higher -> brown for mountains (Fig. 8).

1.3 A peek into the code

Unfortunately, I don’t expect it to be useful without a proper explanation, but nonetheless I’ll present a small snippet of the Main module of the procedural planets game. Additionally, the full source is avaliable¹ – that’s also where engine development is happening. The next feature I’ve just completed, at the time of writing, is a gradient editor for the in game GUI (Fig. 1).

As promised, here’s a quick look at the Main module of the procedural planets game:

initG :: Ghengin World ()
initG = do

  -- Planet settings used to generate the planet and which are edited through the UI
  ps <- makeSettings @PlanetSettings
  (planetMesh,minmax) <- newPlanet ps

  -- Load the planet gradient texture
  sampler <- createSampler FILTER_NEAREST SAMPLER_ADDRESS_MODE_CLAMP_TO_EDGE
  tex         <- texture "assets/planet_gradient.png" sampler

  -- Create the render pipeline based on the shader definition
  planetPipeline <- makeRenderPipeline Shader.shaderPipeline

  -- Create a material which will be validated against the render pipeline at compile time
  m1 <- material (Texture2DBinding tex . StaticBinding (vec3 1 0 0) . StaticBinding minmax) planetPipeline

  -- Create a render packet with the mesh, material, and pipeline.
  -- All entities with a RenderPacket component are rendered according to it.
  let p1 = renderPacket planetMesh m1 planetPipeline

  -- Define our scene graph
  sceneGraph do

    -- A planet entity, with the planet render packet and a transform
    e1 <- newEntity ( p1, Transform (vec3 0 0 0) (vec3 1 1 1) (vec3 0 (pi/2) 0) )

    -- A camera
    newEntity ( Camera (Perspective (radians 65) 0.1 100) ViewTransform
              , Transform (vec3 0 0 0) (vec3 1 1 1) (vec3 0 0 0))

    -- The planet UI component based on the `ps` settings
    newEntityUI "Planet"  $ makeComponents ps (e1,tex)

  pure ()

updateG :: () -> DeltaTime -> Ghengin World Bool
updateG () dt = do

  -- Every frame we update the first person camera with the user inputs and the planet's rotation
  cmapM $ \(_ :: Camera, tr :: Transform) -> updateFirstPersonCameraTransform dt tr
  cmap $ \(_ :: RenderPacket, tr :: Transform) -> (tr{rotation = withVec3 tr.rotation (\x y z -> vec3 x (y+0.5*dt) z) } :: Transform)

  pure False

main :: IO ()
main = do
  -- Run the game with this init, update and end function
  ghengin w initG undefined updateG endG

• 1 Symbolic Maths in E-graphs
  • 1.1 Syntax
  • 1.2 Language
  • 1.3 Analysis
• 2 Equality saturation on symbolic expressions
  • 2.1 Cost function
  • 2.2 Rewrite rules
  • 2.3 Equality saturation, finally

hegg is a Haskell-native library providing fast e-graphs and equality saturation, based on egg: Fast and Extensible Equality Saturation and Relational E-matching.

Suggested material on equality saturation and e-graphs for beginners

• egg: Fast and Extensible Equality Saturation in a 5m video
• egg’s users guide

To get a feel for how we can use hegg and do equality saturation in Haskell, we’ll write a simple numeric symbolic manipulation library that can simplify expressions according to a set of rewrite rules by leveraging equality saturation.

I hope to eventually write a better exposition that assumes less prior knowledge which introduces first e-graphs-only workflows, and only then equality saturation, from a hegg user’s perspective. Until then, this rough tutorial serves as an alternative.

1 Symbolic Maths in E-graphs

If you’ve never heard of symbolic mathematics you might get some intuition from reading Let’s Program a Calculus Student first. First, we define our symbolic maths language and enable it to be represented by an e-graph using hegg.

1.1 Syntax

We’ll start by defining the abstract syntax tree for our simple symbolic expressions:

data SymExpr = Const Double
             | Symbol String
             | SymExpr :+: SymExpr
             | SymExpr :*: SymExpr
             | SymExpr :/: SymExpr
infix 6 :+:
infix 7 :*:, :/:

e1 :: SymExpr
e1 = (Symbol "x" :*: Const 2) :/: (Const 2) -- (x*2)/2

You might notice that (x*2)/2 is the same as just x. Our goal is to get equality saturation to do that for us.

Our second step is to instance Language for our SymExpr

1.2 Language

Language is the required constraint on expressions that are to be represented in e-graph and on which equality saturation can be run:

type Language l = (Traversable l, ∀ a. Ord a => Ord (l a))

To declare a Language we must write the “base functor” of SymExpr (i.e. use a type parameter where the recursion points used to be in the original SymExpr), then instance Traversable l, ∀ a. Ord a => Ord (l a) (we can do it automatically through deriving), and write an Analysis instance for it (see next section).

data SymExpr a = Const Double
               | Symbol String
               | a :+: a
               | a :*: a
               | a :/: a
               deriving (Eq, Ord, Show, Functor, Foldable, Traversable)
infix 6 :+:
infix 7 :*:, :/:

Suggested reading on defining recursive data types in their parametrized version: Introduction To Recursion Schemes

If we now wanted to represent an expression, we’d write it in its fixed-point form

e1 :: Fix SymExpr
e1 = Fix (Fix (Fix (Symbol "x") :*: Fix (Const 2)) :/: (Fix (Const 2))) -- (x*2)/2

Then, we define an Analysis for our SymExpr.

1.3 Analysis

E-class analysis is first described in egg: Fast and Extensible Equality Saturation as a way to make equality saturation more extensible.

With it, we can attach analysis data from a semilattice to each e-class. More can be read about e-class analysis in the Data.Equality.Analsysis module and in the paper.

We can easily define constant folding (2+2 being simplified to 4) through an Analysis instance.

An Analysis is defined over a domain and a language. To define constant folding, we’ll say the domain is Maybe Double to attach a value of that type to each e-class, where Nothing indicates the e-class does not currently have a constant value and Just i means the e-class has constant value i.

instance Analysis (Maybe Double) SymExpr
  makeA = ...
  joinA = ...
  modifyA = ...

Let’s now understand and implement the three methods of the analysis instance we want.

makeA is called when a new e-node is added to a new e-class, and constructs for the new e-class a new value of the domain to be associated with it, always by accessing the associated data of the node’s children data. Its type is l domain -> domain, so note that the e-node’s children associated data is directly available in place of the actual children.

We want to associate constant data to the e-class, so we must find if the e-node has a constant value or otherwise return Nothing:

makeA :: SymExpr (Maybe Double) -> Maybe Double
makeA = \case
  Const x -> Just x
  Symbol _ -> Nothing
  x :+: y -> (+) <$> x <*> y
  x :*: y -> (*) <$> x <*> y
  x :/: y -> (/) <$> x <*> y

joinA is called when e-classes c1 c2 are being merged into c. In this case, we must join the e-class data from both classes to form the e-class data to be associated with new e-class c. Its type is domain -> domain -> domain. In our case, to merge Just _ with Nothing we simply take the Just, and if we merge two e-classes with a constant value (that is, both are Just), then the constant value is the same (or something went very wrong) and we just keep it.

joinA :: Maybe Double -> Maybe Double -> Maybe Double
joinA Nothing (Just x) = Just x
joinA (Just x) Nothing = Just x
joinA Nothing Nothing  = Nothing
joinA (Just x) (Just y) = if x == y then Just x else error "ouch, that shouldn't have happened"

Finally, modifyA describes how an e-class should (optionally) be modified according to the e-class data and what new language expressions are to be added to the e-class also w.r.t. the e-class data. Its type is ClassId -> EGraph domain l -> EGraph domain l, where the first argument is the id of the class to modify (the class which prompted the modification), and then receives and returns an e-graph, in which the e-class has been modified. For our example, if the e-class has a constant value associated to it, we want to create a new e-class with that constant value and merge it to this e-class.

-- import Data.Equality.Graph.Lens ((^.), _class, _data)
modifyA :: ClassId -> EGraph (Maybe Double) SymExpr -> EGraph (Maybe Double) SymExpr
modifyA c egr
    = case egr ^._class c._data of
        Nothing -> egr
        Just i ->
          let (c', egr') = represent (Fix (Const i)) egr
           in snd $ merge c c' egr'

Modify is a bit trickier than the other methods, but it allows our e-graph to change based on the e-class analysis data. Note that the method is optional and there’s a default implementation for it which doesn’t change the e-class or adds anything to it. Analysis data can be otherwise used, e.g., to inform rewrite conditions.

By instancing this e-class analysis, all e-classes that have a constant value associated to them will also have an e-node with a constant value. This is great for our simple symbolic library because it means if we ever find e.g. an expression equal to 3+1, we’ll also know it to be equal to 4, which is a better result than 3+1 (we’ve then successfully implemented constant folding).

If, otherwise, we didn’t want to use an analysis, we could specify the analysis domain as () which will make the analysis do nothing, because there’s an instance polymorphic over lang for () that looks like this:

instance Analysis () lang where
  makeA _ = ()
  joinA _ _ = ()

2 Equality saturation on symbolic expressions

Equality saturation is defined as the function

equalitySaturation :: forall l. Language l
                   => Fix l             -- ^ Expression to run equality saturation on
                   -> [Rewrite l]       -- ^ List of rewrite rules
                   -> CostFunction l    -- ^ Cost function to extract the best equivalent representation
                   -> (Fix l, EGraph l) -- ^ Best equivalent expression and resulting e-graph

To recap, our goal is to reach x starting from (x*2)/2 by means of equality saturation.

We already have a starting expression, so we’re missing a list of rewrite rules ([Rewrite l]) and a cost function (CostFunction).

2.1 Cost function

Picking up the easy one first:

type CostFunction l cost = l cost -> cost

A cost function is used to attribute a cost to representations in the e-graph and to extract the best one. The first type parameter l is the language we’re going to attribute a cost to, and the second type parameter cost is the type with which we will model cost. For the cost function to be valid, cost must instance Ord.

We’ll say Consts and Symbols are the cheapest and then in increasing cost we have :+:, :*: and :/:, and model cost with the Int type.

cost :: CostFunction SymExpr Int
cost = \case
  Const  x -> 1
  Symbol x -> 1
  c1 :+: c2 -> c1 + c2 + 2
  c1 :*: c2 -> c1 + c2 + 3
  c1 :/: c2 -> c1 + c2 + 4

2.2 Rewrite rules

Rewrite rules are transformations applied to matching expressions represented in an e-graph.

We can write simple rewrite rules and conditional rewrite rules, but we’ll only look at the simple ones.

A simple rewrite is formed of its left hand side and right hand side. When the left hand side is matched in the e-graph, the right hand side is added to the e-class where the left hand side was found.

data Rewrite lang = Pattern lang := Pattern lang          -- Simple rewrite rule
                  | Rewrite lang :| RewriteCondition lang -- Conditional rewrite rule

A Pattern is basically an expression that might contain variables and which can be matched against actual expressions.

data Pattern lang
    = NonVariablePattern (lang (Pattern lang))
    | VariablePattern Var

A patterns is defined by its non-variable and variable parts, and can be constructed directly or using the helper function pat and using OverloadedStrings for the variables, where pat is just a synonym for NonVariablePattern and a string literal "abc" is turned into a Pattern constructed with VariablePattern.

We can then write the following very specific set of rewrite rules to simplify our simple symbolic expressions.

rewrites :: [Rewrite SymExpr]
rewrites =
  [ pat (pat ("a" :*: "b") :/: "c") := pat ("a" :*: pat ("b" :/: "c"))
  , pat ("x" :/: "x")               := pat (Const 1)
  , pat ("x" :*: (pat (Const 1)))   := "x"
  ]

2.3 Equality saturation, finally

We can now run equality saturation on our expression!

let expr = fst (equalitySaturation e1 rewrites cost)

And upon printing we’d see expr = Symbol "x"!

If we had instead e2 = Fix (Fix (Fix (Symbol "x") :/: Fix (Symbol "x")) :+: (Fix (Const 3))) -- (x/x)+3, we’d get expr = Const 4 because of our rewrite rules put together with our constant folding!

This was a first introduction which skipped over some details but that tried to walk through fundamental concepts for using e-graphs and equality saturation with this library.

The final code for this tutorial is available under test/SimpleSym.hs

A more complicated symbolic rewrite system which simplifies some derivatives and integrals was written for the testsuite. It can be found at test/Sym.hs.

This library could also be used not only for equality-saturation but also for the equality-graphs and other equality-things (such as e-matching) available. For example, using just the e-graphs from Data.Equality.Graph to improve GHC’s pattern match checker (https://gitlab.haskell.org/ghc/ghc/-/issues/19272).

• 1 Functional Reactive Programming
  • 1.1 Behaviours
  • 1.2 Events
• 2 Reflex
  • 2.1 Building UIs with Reflex-Dom
  • 2.2 Reflex Combinators
• 3 Example: 101companies

1 Functional Reactive Programming

Functional reactive programming is a general paradigm well suited to programming real-time systems in a high-level and functional way.

Real-time systems or reactive systems are those that handle continuous time-varying values, discrete events in real time, and react accordingly. A good example of these systems is a mobile robot. They must take into consideration continuous inputs like wheel speed, orientation, and discrete events such as detection of another object.

The class of reactive systems we’re interested in here is interactive graphical UIs. A user interface has multiple components that can be seen as discrete events and continuous time-varying values. An input box, where one might write their name, is an example of a time varying value (it continously changes – whenever the user types something); a button is an example of a discrete event in time: at certain points in time the user will click the button. This will make more sense with practice and code samples.

So the promise of functional reactive programming is that we can program these complicated reactive systems in a pure functional way. But how?

Functional Reactive Programming introduces two key concepts: Behaviours and Events.

• Behaviours are first-class values that vary over continuous time.

  That means a behaviour is a value that changes with time and can be passed to/returned by functions.

• Events are first-class values that occur at some points in time.

  They may refer, e.g., to happenings in the real world time – such as a mouse click or a key press.

And then, it says that the FRP implementation will handle all time-related details so that the programmer can describe their reactive system without thinking about what happens at any particular point in time, but rather thinking about what happens accross all points in time.

This type of wholemeal programming (working with the whole, not the individual part) is quite common in functional programming. When working with lists, the same pattern appears: we think and define functions in terms of entire lists, rather than about any particular element in the list.

Functional reactive programming just takes it further by abstracting over time itself.

In the end, the programmer doesn’t need to worry about any of the how, just the what (which quite nicely aligns with the original paper on FRP[1], which distinguishes presentation from modelling). So, we manipulate time-dependent behaviours and events, but we never directly see the time.

In fact, both Behaviors and Events are usually abstract data types, meaning we don’t even need to know how they are defined.

Later we’ll see what this actually looks like.

• [1] Functional Reactive Animation, …

1.1 Behaviours

A Behaviour is a first-class value that varies over continuous time. In other words, a Behaviour is a value that can be passed to and returned by functions (is first-class) and this value is defined for all time (and can change accross time)

Intuitively, it can be seen as a function from time to value (b :: Time -> Value) and could be visualized as:

Note how this function is pure (there are no side effects): we simply map all points in time to a value.

A behaviour is an abstract type constructor, meaning we know nothing about its data constructors. We interact with behaviours through the combinators and functions defined by the FRP implementation we’re using. The type parameter of Behavior indicates the type of the values.

data Behavior a

Behaviors are usually Functors (it depends on the FRP implementation, in Reflex they are). We can imagine that if we have a behaviour (x :: Behavior Int) which is for all points in time equal to 1 (you might imagine the 2D graph of the constant math function f(x) = 1), there is an easy way to define a behavior y which is constantly equal to 2: fmap (+1) x :: Behavior Int.

Let’s write a short example of a function that transforms a Behavior String into a Behavior (Maybe Color)

data Color = Yellow | Magenta | Cyan

color :: Behavior String -> Behavior (Maybe Color)
color bstr = fmap toColor bstr
  where
    toColor "yellow"  = Just Yellow
    toColor "magenta" = Just Magenta
    toColor "cyan"    = Just Cyan
    toColor _         = Nothing

An example of a behaviour we’ll use is the value in an input text box. This box has a string at all times. Before the user writes anything, the value is the empty string. The value at some time after the user has written “123” is “123”.

1.2 Events

An Event is a first-class value that occurs at discrete points in time. In other words, an Event is a value that can be passed to and returned by functions (is first-class) and this value is defined at only some points in time. If the event is occuring, and if it is, what its value is.

It’s a bit harder to think about intuitively, but one can imagine an event as a function from time to maybe a value (e :: Time -> Maybe Value) or as a list of values and the times at which they occur (e :: [(Time, Value)]).

An event is also an abstract type constructor, meaning we know nothing about its data constructors. As with behaviors, we interact with events through the combinators and functions defined by the FRP implementation we’re using. The type parameter of Event also indicates the type of the values.

data Event a

Above, a is the type of the value the event carries when it occurs.

Events are functors! You can think of them being functors the same way you would about Behaviors.

An example of an event we’ll use is the one generated by a button. Everytime the button is clicked, an event occurs of type Event () (a button click has no associated value, unlike, e.g., key presses, which would have type Event Char).

Ahead we’ll see how we can manipulate and put together events and behaviors to build a reactive application.

2 Reflex

Reflex is a haskell library for functional reactive programming in haskell.

Interactive programs without callbacks or side-effects. Functional Reactive Programming (FRP) uses composable events and time-varying values to describe interactive systems as pure functions. Just like other pure functional code, functional reactive code is easier to get right on the first try, maintain, and reuse.

Reflex is a fully-deterministic, higher-order Functional Reactive Programming interface and an engine that efficiently implements that interface.

Reflex provides the above described abstractions (behaviors and events), functions to work with them, and an additional abstraction called Dynamics. For our intents and purposes, we’ll consider dynamics to be the same as behaviors, since discussing the details of it would be out of the scope of this tutorial[2].

• [2] In a very short note: Reflex is implemented with push-pull evaluation, and thus often requires explicit push notifications to update things like the DOM. Event push notifications when they occur, but Behaviors don’t push anything. A Dynamic is a combination of an Event and Behavior, meaning it’s a continuous time-varying value that pushes notification whenever it changes, and fills the spaces where we need a behaviour and to know when the value changes.

2.1 Building UIs with Reflex-Dom

To build a UI we’ll use reflex-dom which is a haskell library that provides a monadic interface for building a DOM-based UI with functional reactive programming abstractions from reflex.

A DOM user interface is defined with element tags. A webpage can have many elements such as divisions and text paragraphs.

<div>
    <p>This is a paragraph.</p> 
    <button>Button below the paragraph</button>
</div>

<div>
    <p>This is the second division.</p>
    <input></input>
</div>

Would define a webpage with two divisions (<div>). The first division has a paragraph (<p>) and a button (<button>), the second, a paragraph and and a text-box for input (<input>).

Reflex-dom defines the Widget monad[3] and multiple functions with which we can create our UI. However, some of these functions do more than build the UI: they return and accept events and behaviors! Let’s look at a few.

We can put text on the UI, without creating a new element. Notice how the returned Widget is a computation that builds the UI and the resulting value of the said computation has type ().

text :: Text -> Widget ()

The most basic element-creating function is el. It takes the name of the element as a string, a Widget computation, and returns another Widget computation. This still does nothing regarding FRP.

el :: Text -> Widget a -> Widget a

ex1 = el "p" (text "This is a paragraph")
-- would correspond to
-- <p>This is a paragraph</p>

ex2 = el "div" (el "p" (text "Other paragraph"))
-- would correspond to
-- <div>
--      <p>This is a paragraph</p>
-- </div>

ex3 = el "div" $ do
        el "p" (text "First paragraph")
        el "p" (text "Second paragraph")
-- would correspond to
--  <div>
--      <p>First paragraph</p>
--      <p>Second paragraph</p>
--  <div>

A button can be clicked at some points in time. We can model a button click with an event! There is a convenient function that returns a Widget computation that creates a button and returns an event that occurs when the button is clicked. We’ll see ahead an example in which we this event.

button :: Text -> Widget (Event t ()) 

ex4 = el "div" $ do
        el "p" (text "Before button")
        clickEvt <- button "Click me!"
        el "p" (text "After button")
-- would correspond to
--  <div>
--      <p>Before button</p>
--      <button>Click me!</button>
--      <p>After button</p>
--  <div>

An input text box has a continuous time-varying value: What the user has input in the box. We can model the value inside the input box with a behavior! As with the button, we already have a function that returns a Widget computation that creates an input box and which returns a behavior of the value in the text box.

input :: Widget (Dynamic Text)

ex5 = el "div" $ do
        inputBehavior <- input
        return ()

-- would correspond to
--  <div>
--      <input></input>
--  <div>

Lastly, we have a function for displaying a behavior of a string. Whatever the value of the string is at the current time, is what’s displayed on the string. Non-surprisingly, this function is called dynText.

dynText :: Dynamic Text -> Widget ()

ex6 = el "div" $ do
        inputBehavior <- input
        el "p" (dynText inputBehavior)
-- what would this look like?
-- <div>
--      <input></input>
--      <p>???</p>
-- </div>
-- not so simple, would require javascript

Next we’ll look at some UI-independent combinators on events and behaviors, and an example of how we can use them in building a more complex UI.

• [3] In fact, Widget x is the monad (Widget :: * -> * -> *), where x is a type parameter to guarantee contexts don’t get mixed.

2.2 Reflex Combinators

In this section we describe some reflex combinators for manipulating events and behaviors. Remember that for our purposes we’ll consider dynamics to be the same as behaviors but with a different name.

We can create a behavior which is constant over all time, given the constant value.

constDyn :: a -> Dynamic a

d1 = constDyn 2 -- is 2 at all points in time

We can create a behavior that changes to the value of an event every time said event occurs, provided an initial value for the behavior to have before the first occurrence of the event is given.

holdDyn :: a -> Event a -> Widget (Dynamic a)

d2 = holdDyn 'A' keyPress -- until the first event occurrence is 'A', and then it's the value of the event occurrence
    where keyPress :: Event Char

We can create a behavior that changes everytime an event occurs, provided the function to fold the event value with the current value, and an initial value. This is somewhat similar to a list fold, but it’s a actually a fold across time.

foldDyn :: (a -> b -> b) -> b -> Event a -> m (Dynamic b)

-- Note: foldr :: (a -> b -> b) -> b -> [a] -> b

As an example, we can now define an application that displays a button and the number of times said button has been clicked (remember how behaviors instance Functor)

app = el "div" $ do
        clickEvt <- button "Click me!"
        clickAmount <- foldDyn (\_ acc -> acc+1) 0 clickEvt
        el "p" (dynText (displayAmt <$> clickAmount))
    where
        displayAmt x = "Clicked " <> showT x <> " times."

-- Note how length for normal functions can be defined as
length ls = foldr (\_ acc -> acc+1) 0 ls

This might be a bit confusing so we’ll walk through each value.

• clickEvt has type Event (), representing the occurrences of a button click through time
• clickAmount has type Dynamic Int. We start with the value 0, and when the Event () occurs, we apply the lambda of type () -> Int -> Int which ignores the first argument and adds +1 to the existing counter.
• displayAmt <$> clickAmount has type Dynamic Text, meaning we can display it with dynText. Note that displayAmt :: Int -> Text, and we fmap it over Dynamic Int (hence why we get Dynamic Text).

If you were something similar[4] to this code, you’d see an application with a button and a text paragraph with a counter. Every time you click the button the counter will increase.

Lastly, we’ll cover a function that allows us to sample the value of a behavior every time an event occurs.

tagPromptlyDyn :: Dynamic a -> Event b -> Event a

-- e1 will occur with the value of the input box every time the button is clicked
e1 :: Event Text
e1 = do
    click <- button "Sample input box value"
    valBehv <- input
    return (tagPromptlyDyn valBehv click)

Next we’ll build a simple application for the Company example.

• [4] A full runnable example of this program is provided in the appendix(?)

3 Example: 101companies

For our 101 companies example we’ll build a simple application with three distinct divisions. One for information where we display the total price of the employees, one for the hiring form with which we can add employees to our company, and one with the list of the employees in the company.

Our application models and displays the changes in a company over time. We’ll model the company with a behavior. Initially, the company will be the sample company. Whenever an employee is hired the company behavior changes accordingly.

The first section is a simple division with a text that changes according to a behavior. We only want to display the total value of a company that changes over time, so we’ll capture this idea with this function:

info :: Dynamic Company -> Widget x ()
info c =
    el "div" $ do
        el "h3" (text "Info")
        dynText (companyToTotal <$> c)
  where
    companyToTotal :: Company -> Text
    companyToTotal c = "Total cost: $" <> showT (total c)

Our hiring department defines a company that changes over time: It defines the button that will add an employee to the company everytime it’s clicked, and otherwise doesn’t take any arguments.

hiring :: Widget x (Dynamic Company)
hiring =
    el "div" do
        el "h3" (text "Hiring")
        name    <- input "Name"
        address <- input "Addr."
        salary  <- input "Salary"
        clickEv <- button "Hire"
        let behaviorEmployee :: Dynamic Employee = Employee <$> name <*> address <*> (readSalary <$> salary)
        let hireEmployee     :: Event Employee = tagPromptlyDyn behaviorEmployee clickEv
        foldDyn addEmployee sampleCompany hireEmployee

  where
    addEmployee :: Employee -> Company -> Company
    addEmployee e (Company n es) = Company n (e:es)

    readSalary :: Text -> Salary
    readSalary = read . unpack

• input is a helper function defined in our module with type Text -> Widget x (Dynamic Text). It takes a text to display above the input, and returns the DOM-building computation that returns a behavior of the value inside the input box.
• We make three input boxes and get three behaviors of text.
• We make a button and get clickEvt :: Event ().
• We make a new behavior behaviorEmployee that combines the continuos values in the input boxes to define an employee that changes over continuous time. (We make use of applicative functor syntax here)
• We create a new event hireEmployee by combining the click event with the employee behavior. This event fires every time the button is clicked, and the value will be the current employee described in the input boxes at that time.
• In the last line, we return the behavior of the company by folding (foldDyn) over the employee-creating event hireEmployee and starting with the initial company sampleCompany. That is, every time the event fires, we apply addEmployee to the value of the event and to the current value of the behavior (which initially is equal to sampleCompany).

Finally, we’ll display the list of employees. We’ll make use of a reflex function called simpleList :: Dynamic t [v] -> (Dynamic t v -> Widget a) -> Widget x ()[5]. It basically says that with a behavior of a list of values, and with a function that can turn the behavior of each value in the list into a widget, we get a widget which applies the function to every element of the list.

employeesList :: Dynamic Company -> Widget x ()
employeesList c =
    section do
        el "h3" (text "Employees")
        el "table" do
            el "tr" do
                el "td" (text "Name")
                el "td" (text "Address")
                el "td" (text "Salary")
            simpleList (employees <$> c) \dEmployee -> do
                el "tr" do
                    let dName = fmap emplName dEmployee
                    let dAddr = fmap emplAddr dEmployee
                    let dSalr = fmap emplSalr dEmployee
                    el "td" (dynText dName)
                    el "td" (dynText dAddr)
                    el "td" (dynText dSalr)
        return ()
            where
                emplName (Employee n _ _) = n
                emplAddr (Employee _ a _) = a
                emplSalr (Employee _ _ s) = "$" <> (pack . show) s

• We’re creating an HTML table. It consists of table rows (<tr>) and each table row has multiple cells called table data (<td>).
• We display a row with the headers Name, Address, and Salary.
• We apply the simple list function to the list of employees in the company. Note how c :: Dynamic Company, and employees :: Company -> [Employee], so employees <$> c :: Dynamic [Employee].
• The function to display each element of the list displays the name, address, and salary of the employee in a table row. We get the behavior of each individual attribute (name, salary, employee) by using fmap on the behavior of the employee with the utility functions defined below.

Finally, we put together these three things in one widget:

body :: Widget x ()
body = mdo
    info dynamicCompany

    dynamicCompany <- hiring

    employeesList dynamicCompany

hiring defines the behavior of the company, and both info and employeesList consume it. We then define main using a main reflex-dom function to run the body of the application:

main :: IO ()
main = mainWidgetWithCss style body

style = "body { padding: 2em; margin: auto; max-width: 50em; } h3 {..."

You might note that we pass dynamicCompany to info before it’s actually defined. I think we’ve seen enough mind-blowing things for this post, but in short, we’re making use of an extension called RecursiveDo that allows us to do these logic-defying things.

• [5] Simplified version of the signature for our purposes… The actual one is simpleList :: (Adjustable t m, MonadHold t m, PostBuild t m, MonadFix m) => Dynamic t [v] -> (Dynamic t v -> m a) -> m (Dynamic t [a])