Calling Haskell from Swift

Apr 2, 2024

Crossing the language boundary between Haskell and Swift. This is the second part of an in-depth guide into developing native applications using Haskell with Swift.

Contents

This is the second installment of the in-depth series of blog-posts on developing native macOS and iOS applications using both Haskell and Swift/SwiftUI. This post covers how to call (non-trivial) Haskell functions from Swift by using a foreign function calling-convention strategy similar to that described by Calling Purgatory from Heaven: Binding to Rust in Haskell that requires argument and result marshaling.

You may find the other blog posts in this series interesting:

  1. Creating a macOS app with Haskell and Swift

The series of blog posts is further accompanied by a github repository where each commit matches a step of this tutorial. If in doubt regarding any step, check the matching commit to make it clearer.

This write-up has been cross-posted to Well-Typed’s Blog.

1 Introduction

We’ll pick up from where the last post ended – we have set up an XCode project that includes our headers generated from Haskell modules with foreign exports and linking against the foreign library declared in the cabal file. We have already been able to call a very simple Haskell function on integers from Swift via Haskell’s C foreign export feature and Swift’s C interoperability.

This part concerns itself with calling idiomatic Haskell functions, which typically involve user-defined datatypes as inputs and outputs, from Swift. Moreover, these functions should be made available to Swift transparently, such that Swift calls them as it does other idiomatic functions, with user defined structs and classes.

For the running example, the following not-very-interesting function will suffice to showcase the method we will use to expose this function from Haskell to Swift, which easily scales to other complex data types and functions.

data User
  = User { name :: String
         , age  :: Int
         }

birthday :: User -> User
birthday user = user{age = user.age + 1}

The Swift side should wrap Haskell’s birthday:

struct User {
    let name: String
    let age: Int
}

// birthday(user: User(name: "Anton", age: 33)) = User(name: "Anton", age: 34)
func birthday(user: User) -> User {
    // Calls Haskell function...
}

To support this workflow, we need a way to convert the User datatype from Haskell to Swift, and vice versa. We are going to serialize (most) inputs and outputs of a function. Even though the serialization as it will be described may seem complex, it can be automated with Template Haskell and Swift Macros and packed into a neat interface – which I’ve done in haskell-swift.

As a preliminary step, we add the User data type and birthday function to haskell-framework/src/MyLib.hs, and the Swift equivalents to SwiftHaskell/ContentView.swift from the haskell-x-swift-project-steps example project.

2 Marshaling Inputs and Outputs

Marshaling the inputs and outputs of a function, from the Swift perspective, means to serialize the input values into strings, and receive the output value as a string which is then decoded into a Swift value. The Haskell perspective is dual.

Marshaling/serializing is a very robust solution to foreign language interoperability. While there is a small overhead of encoding and decoding at a function call, it almost automatically extends to, and enables, all sorts of data to be transported across the language boundary, without it being vulnerable to compiler implementation details and memory representation incompatibilities.

We will use the same marshaling strategy that Calling Purgatory from Heaven: Binding to Rust in Haskell does. In short, the idiomatic Haskell function is wrapped by a low-level one which deserializes the Haskell values from the argument buffers, and serializes the function result to a buffer that the caller provides. More specifically,

We will use JSON as the serialization format: this choice is motivated primarily by convenience because Swift can derive JSON instances for datatypes out of the box (without incurring in extra dependencies), and in Haskell we can use aeson to the same effect. In practice, it could be best to use a format such as CBOR or Borsh which are binary formats optimised for compactness and serialization performance.

2.1 Haskell’s Perspective

Extending the User example requires User to be decodable, which can be done automatically by adding to the User declaration:

deriving stock Generic
deriving anyclass (ToJSON, FromJSON)

With the appropriate extensions and importing the necessary modules in MyLib:

{-# LANGUAGE DerivingStrategies, DeriveAnyClass #-}

-- ...

import GHC.Generics
import Data.Aeson

The MyForeignLib module additionally must import

import Foreign.Ptr
import Foreign.Storable
import Foreign.Marshal
import Data.Aeson
import Data.ByteString
import Data.ByteString.Unsafe

Now, let’s (foreign) export a function c_birthday that wraps birthday above in haskell-framework/flib/MyForeignLib.hs, using the described method.

First, the type definition of the function receives the buffer with the User argument, and a buffer to write the User result to. We cannot use tuples because they are not supported in foreign export declarations, but the intuition is that the first two arguments represent the original User input, and the two latter arguments represent the returned User.

c_birthday :: Ptr CChar -> Int -> Ptr CChar -> Ptr Int -> IO ()

Then, the implementation – decode the argument, encode the result, write result size to the given memory location and the result itself to the buffer, if it fits.

c_birthday cstr clen result size_ptr = do

We transform the (Ptr CChar, Int) pair into a ByteString using unsafePackCStringLen, and decode a User from the ByteString using decodeStrict:

  -- (1) Decode C string
  Just user <- decodeStrict <$> unsafePackCStringLen (cstr, clen)

We apply the original birthday function to the decoded user. In our example, this is a very boring function, but in reality this is likely a complex idiomatic Haskell function that we want to expose to.

  -- (2) Apply `birthday`
  let user_new = birthday user

We encode the new_user :: User as a ByteString, and use unsafeUseAsCStringLen to get a pointer to the bytestring data and its length. Finally, we get the size of the result buffer, write the actual size of the result to the given memory location, and, if the actual size fits the buffer, copy the bytes from the bytestring to the given buffer.

  -- (3) Encode result
  unsafeUseAsCStringLen (toStrict $ encode user_new) $ \(ptr,len) -> do

    -- (3.2) What is the size of the result buffer?
    size_avail <- peek size_ptr

    -- (3.3) Write actual size to the int ptr.
    poke size_ptr len

    -- (3.4) If sufficient, we copy the result bytes to the given result buffer
    if size_avail < len
       then do
         -- We need @len@ bytes available
         -- The caller has to retry
         return ()
       else do
         moveBytes result ptr len

If the written required size is larger than the given buffer, the caller will retry.

Of course, we must export this as a C function.

foreign export ccall c_birthday :: Ptr CChar -> Int -> Ptr CChar -> Ptr Int -> IO ()

This makes the c_birthday function wrapper available to Swift in the generated header and at link time in the dynamic library.

2.2 Swift’s Perspective

In Swift, we want to be able to call the functions exposed from Haskell via their C wrappers from a wrapper that feels idiomatic in Swift. In our example, that means wrapping a call to c_birthday in a new Swift birthday function.

In ContentView.swift, we make User JSON-encodable/decodable by conforming to the Codable protocol:

struct User: Codable {
    // ...
}

Then, we implement the Swift side of birthday which simply calls c_birthday – the whole logic of birthday is handled by the Haskell side function (recall that birthday could be incredibly complex, and other functions exposed by Haskell will indeed be).

func birthday(user: User) -> User {
    // ...
}

Note: in the implementation, a couple of blocks have to be wrapped with a do { ... } catch X { ... } but I omit them in this text. You can see the commit relevant to the Swift function wrapper implementation in the repo with all of these details included.

First, we encode the Swift argument into JSON using the Data type (plus its length) that will serve as arguments to the foreign C function.

let enc = JSONEncoder()
let dec = JSONDecoder()

var data: Data = try enc.encode(user)
let data_len = Int64(data.count)

However, a Swift Data value, which represents the JSON as binary data, cannot be passed directly to C as a pointer. For that, we must use withUnsafeMutableBytes to get an UnsafeMutableRawBufferPointer out of the Data – that we can pass to the C foreign function. withUnsafeMutableBytes receives a closure that uses an UnsafeMutableRawBufferPointer in its scope and returns the value returned by the closure. Therefore we can return the result of calling it on the user Data we encoded right away:

return data.withUnsafeMutableBytes { (rawPtr: UnsafeMutableRawBufferPointer) in
    // here goes the closure that can use the raw pointer,
    // the code for which we describe below
}

We allocate a buffer for the C foreign function to insert the result of calling the Haskell function, and also allocate memory to store the size of the buffer. We use withUnsafeTemporaryAllocation to allocate a buffer that can be used in the C foreign function call. As for withUnsafeMutableBytes, this function also takes a closure and returns the value returned by the closure:

// The data buffer size
let buf_size = 1024048 // 1024KB

// A size=1 buffer to store the length of the result buffer
return withUnsafeTemporaryAllocation(of: Int.self: 1) { size_ptr in
    // Store the buffer size in this memory location
    size_ptr.baseAddress?.pointee = buf_size

    // Allocate the buffer for the result (we need to wrap this in a do { ...} catch for reasons explained below)
    do {
        return withUnsafeTemporaryAllocation(byteCount: buf_size, alignment:1) { res_ptr in

            // Continues from here ...
        }
    } catch // We continue here in due time ...
}

We are now nested deep within 3 closures: one binds the pointer to the argument’s data, the other the pointer to the buffer size, and the other the result buffer pointer. This means we can now call the C foreign function wrapping the Haskell function:

c_birthday(rawPtr.baseAddress, data_len, res_ptr.baseAddress, size_ptr.baseAddress)

Recalling that the Haskell side will update the size pointed to by size_ptr to the size required to serialize the encoded result, we need to check if this required size exceeds the buffer we allocated, or read the data otherwise:

if let required_size = size_ptr.baseAddress?.pointee {
    if required_size > buf_size {
        // Need to try again
        throw HsFFIError.requiredSizeIs(required_size)
    }
}

return dec.decode(User.self, from: Data(bytesNoCopy: res_ptr.baseAddress!,
                    count: size_ptr.baseAddress?.pointee ?? 0, deallocator: .none))

where HsFFIError is a custom error defined as

enum HsFFIError: Error {
    case requiredSizeIs(Int)
}

We must now fill in the catch block to retry the foreign function call with a buffer of the right size:

} catch HsFFIError.requiredSizeIs(let required_size) {
    return withUnsafeTemporaryAllocation(byteCount: required_size, alignment:1)
    { res_ptr in
        size_ptr.baseAddress?.pointee = required_size
        c_birthday(rawPtr.baseAddress, data_len, res_ptr.baseAddress, size_ptr.baseAddress)

        return dec.decode(User.self, from: Data(bytesNoCopy: res_ptr.baseAddress!,
                    count: size_ptr.baseAddress?.pointee ?? 0, deallocator: .none))
    }
}

That seems like a lot of work to call a function from Haskell! However, despite this being a lot of code, not a whole lot is happening: we simply serialize the argument, allocate a buffer for the result, and deserialize the result into it. In the worst case, if the serialized result does not fit (the serialized data has over 1M characters), then we naively compute the function a second time (it should not be terribly complicated to avoid this work by caching the result and somehow resuming the serialization with the new buffer). Furthermore, there is a lot of bureocracy in getting the raw pointers to send off to Haskell land – the good news is that all of this can be automated away behind automatic code generation with Template Haskell and Swift Macros.

Expand for the complete function 
func birthday (user : User) -> User {
    let enc = JSONEncoder()
    let dec = JSONDecoder()
    do {
        var data : Data = try enc.encode(user)
        let data_len = Int64(data.count)
        return try data.withUnsafeMutableBytes { (rawPtr:UnsafeMutableRawBufferPointer) in

            // Allocate buffer for result
            let buf_size = 1024000

            return try withUnsafeTemporaryAllocation(of: Int.self, capacity: 1) { size_ptr in
                size_ptr.baseAddress?.pointee = buf_size

                do {
                    return try withUnsafeTemporaryAllocation(byteCount: buf_size, alignment: 1) {
 res_ptr in

                        c_birthday(rawPtr.baseAddress, data_len, res_ptr.baseAddress, size_ptr.baseAddress)

                        if let required_size = size_ptr.baseAddress?.pointee {
                            if required_size > buf_size {
                                throw HsFFIError.requiredSizeIs(required_size)
                            }
                        }
                        return try dec.decode(User.self, from: Data(bytesNoCopy: res_ptr.baseAddress!, count: size_ptr.baseAddress?.pointee ?? 0, deallocator: .none))
                    }
                } catch HsFFIError.requiredSizeIs(let required_size) {
                    print("Retrying with required size: \(required_size)")
                    return try withUnsafeTemporaryAllocation(byteCount: required_size, alignment:
 1) { res_ptr in
                        size_ptr.baseAddress?.pointee = required_size

                        c_birthday(rawPtr.baseAddress, data_len, res_ptr.baseAddress, size_ptr.baseAddress)

                        return try dec.decode(User.self, from: Data(bytesNoCopy: res_ptr.baseAddress!, count: size_ptr.baseAddress?.pointee ?? 0, deallocator: .none))
                    }
                }
            }
        }
    } catch {
        print("Error decoding JSON probably: \(error)")
        return User(name: "", age: 0)
    }
}

We can test that this is working by replacing ContentView with:

struct ContentView: View {
    var body: some View {
        VStack {
            let user = birthday(user: User(name: "Ellie", age: 24))
            Text("Post-birthday, \(user.name) is: \(user.age)!")
        }
        .padding()
    }
}

And you should see:

Fig 1. Swift app displays result of calling idiomatic Haskell function via idiomatic Swift wrapper

3 Metaprogramming at the boundaries

I want to give a quick preview of what is made possible by using compile-time code generation features (Template Haskell in Haskell, Swift Macros in Swift). This foreign function code generation API is exposed by the haskell-swift project, namely the swift-ffi Haskell library and haskell-ffi Swift package (since it is out of the scope of this tutorial, I will not cover how exactly the compile-time code-generation code works, but instead use the API provided by these libraries).

With these top-level foreign interaction facilities, coupled with the build tool also provided by haskell-swift, one can easily bootstrap and develop programs mixing Haskell and Swift.

Let us consider the same example where we define an idiomatic birthday :: User -> User function in Haskell and want to be able to call it from Swift as birthday(user: User) -> User

3.1 Haskell’s perspective

To expose the birthday function to Swift, we simply use the foreignExportSwift Template Haskell function. The whole module could look like this:

{-# LANGUAGE TemplateHaskell #-}
module MyLib where

-- ...
import Swift.FFI

data User
 = User { name :: String
        , age  :: Int
        }
        deriving stock    Generic
        deriving anyclass FromJSON
        deriving anyclass ToJSON

birthday :: User -> User
birthday User{age=x, name=y} = User{age=x+1, name=y}

$(foreignExportSwift 'birthday)

The key bit is the last foreignExportSwift call which will expose a C function with the marshalling-based calling convention we outlined above.

3.2 Swift’s perspective

On the Swift side, we want to use the dual @ForeignImportHaskell macro which generates a Swift function wrapper which in turn invokes the C function exposed by Haskell with the above marshalling strategy. The Swift file could look like:

import HaskellFFI

struct User: Codable {
    let name: String
    let age: Int
}

@ForeignImportHaskell
func birthday(cconv: HsCallJSON, user: User) -> User { stub() }

where birthday could be called e.g. as:

birthday(user: User(name: "Pierre", age: 55))

4 Remarks

The strategy of marshaling for foreign language boundary crossing is very robust and still performant, and is a great fit for the kind of mixed-language application we want to develop robustly.

Even though marshaling is required for robustly traversing the foreign language boundary, I will also explore, in a subsequent post, calling Haskell from Swift by instead coercing the memory representation of a Haskell value into a Swift one – this will mostly be a (very unsafe) and not-at-all robust curiosity, but it will give me an excuse to write a bit about low-level details in Haskell!

In yet another post, I also intend to introduce the hxs tool for bootstrapping Haskell x Swift projects and the libraries that make it so much easier to export Haskell functions and import them from Swift.

The haskell-x-swift-project-steps git repository has a commit matching the steps of this guide, so if anything is unclear you can just let the code speak by itself in checking the commits.

This project, blog post, and research regarding Swift interoperability with Haskell is being partially sponsored by Well-Typed, and is otherwise carried out in my own free time. If you’d also like to sponsor my work on Swift x Haskell interoperability with the goal of developing native macOS/iOS/etc applications, visit my GitHub sponsors page.