feat: Enable sqlite-nio to compile to wasm targets #98
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #98 +/- ##
=======================================
Coverage 70.88% 70.88%
=======================================
Files 10 10
Lines 965 965
=======================================
Hits 684 684
Misses 281 281
🚀 New features to boost your workflow:
|
| #if _pointerBitWidth(_64) | ||
| // We use Int instead of Int64 on 64 bit systems primarily for | ||
| // backwards compatibility, since prior code versions have used Int here. | ||
| public typealias SQLiteDataIntegerType = Int // 64-bit platform, Int = 64 bits | ||
| #elseif _pointerBitWidth(_32) | ||
| public typealias SQLiteDataIntegerType = Int64 // On 32-bit platforms, we want to use 64 bit integers. | ||
| #else | ||
| // If you hit errors here, you may simply need to add | ||
| // a new architectural bit size above (eg. _128) when that | ||
| // exists. | ||
| // | ||
| // Or if the following proposal for pointerBitWidth ever lands in a | ||
| // published Swift version, then the above conditionals may | ||
| // need adjusted: | ||
| // | ||
| // https://forums.swift.org/t/pitch-pointer-bit-width-compile-time-conditional/59572 | ||
| #error("Unsupported integer size") | ||
| #endif |
There was a problem hiding this comment.
sqlite-no has (obviously) not previously supported 32-bit runtime environments. I'm not familiar enough with Wasm to know whether 32-bit is an environment we should be expecting to encounter on a regular basis?
There was a problem hiding this comment.
@gwynne At the moment, the swift for WebAssembly SDK only supports 32 bit. 64 bit support is on the roadmap, but will probably be a while. I'm pretty sure that all the other officially supported swift platforms besides wasm are 32 bit. Except maybe Swift Embedded, but supporting Swift Embedded would likely be a long haul in sqlite-nio.
Package.swift
Outdated
| import PackageDescription | ||
|
|
||
| let wasiPlatform: [Platform] = [.wasi] | ||
| let nonWASIPlatforms: [Platform] = [.macOS, .macCatalyst, .iOS, .tvOS, .watchOS, .visionOS, .driverKit, .linux, .windows, .android, .openbsd] |
There was a problem hiding this comment.
I am really not a fan of having to manually list out every "not that one" platform like this, but I know SwiftPM, ever its eternally helpful self, provides no way of avoiding it (my kingdom for condition: .when(notPlatforms: [...])...).
I'm also not sure how I feel about adding a new third-party dependency here. @0xTim can you chime in on this?
There was a problem hiding this comment.
We definitely can't add a dependency without a stable API yet for sure. Realistically what we should do is migrate SQLiteNIO off NIO entirely to AsyncSequence<ArraySlice<UInt8>> and custom executors. There's nothing that requires NIO anymore with modern Swift Concurrency
There was a problem hiding this comment.
@0xTim @gwynne Agreed, this isn't my favorite way to do this, either. But like Gwynne mentioned, not a lot of better alternatives currently. There isn't currently an allCases option, since each of these are static properties in a struct, rather than an enum. To your point, it is easy for this list to grow out of date. For example, I copied this from the swift-package-manager source a few months ago, and since then .freeBSD was added. As swift grows in popularity, this will be more and more of a problem. I can only assume at some point someone will either change that to an enum, or else add an allCases convenience that centralizes lists like this. A .when(notPlatforms: [...])...) would be really helpful here obviously. Since none of those are options, I'm not sure there is a better way. But I would love to use any better alternative solutions that don't break wasm compilation.
A few things to note here:
- None of the changes in this PR change the publicly advertised supported platforms. Lines 10-13 in the manifest remain untouched.
- This pattern here matches similar approaches used in the swift-nio manifest
The worst case problem I can think of with this implementation is that a platform not included in this list won't get the NIOPosix dependency, and won't be able to compile. Since all of the platforms officially supported sqlite-nio are already included in this list, this would only be a problem for new platforms, and there is no advertised promise of support for those platforms. If it manifests as a problem, it would be trivial to submit a PR to add a new platform to the list.
I suppose I could try to refactor NIO completely out, but that was a bit out of scope of my original goal to support wasm. The current PR changeset should leave existing platform support unaffected. A larger refactor to completely remove NIO and transition to Swift Concurrrency would bring the risk of affecting existing usage of sqlite-nio.
One thing to note, my introduction of nio-async-runtime here is essentially a port to Swift Concurrency. I ran benchmarks comparing performance of nio-async-runtime compared to NIOPosix, and NIOPosix did show significantly better performance in the benchmarks I ran. I'm not sure how it would affect end-to-end performance, but figured I would mention it. There is a risk with transitioning from NIOPosix to Swift Concurrency that performance will decrease. With this PR, that is a non-issue, because all platforms except WASI still use NIOPosix.
Happy to look into any option here, appreciate the great review and feedback! Any thoughts or requests here? 😁
There was a problem hiding this comment.
Just a sanity check on this list, this is also the full list on Swift 5.10. Since the swift-tools-version in this manifest is 5.10, we probably wouldn't want to add the new .freeBSD platform here, since 5.10 versions of SPM wouldn't work. But all of these options are defined in the 5.10 release of SPM
There was a problem hiding this comment.
I refactored this a little to improve the maintenance and clarity. Please let me know if any other changes or improvements can be made.
There was a problem hiding this comment.
@0xTim This now relies on a semver-stable release of nio-async-runtime. Please let me know if there is anything else I can do to improve this PR. 🎉
Package.swift
Outdated
| import PackageDescription | ||
|
|
||
| let wasiPlatform: [Platform] = [.wasi] | ||
| let nonWASIPlatforms: [Platform] = [.macOS, .macCatalyst, .iOS, .tvOS, .watchOS, .visionOS, .driverKit, .linux, .windows, .android, .openbsd] |
There was a problem hiding this comment.
I refactored this a little to improve the maintenance and clarity. Please let me know if any other changes or improvements can be made.
scottmarchant
force-pushed
the
feat/swift-wasm-support-v2
branch
from
e90448c to
908b0bc
Compare
|
Moving this to draft while I upstream NIOAsyncRuntime into swift-nio. |
scottmarchant
force-pushed
the
feat/swift-wasm-support-v2
branch
2 times, most recently
from
de29676 to
b72dc95
Compare
scottmarchant
force-pushed
the
feat/swift-wasm-support-v2
branch
3 times, most recently
from
ef563aa to
aba49b8
Compare
scottmarchant
force-pushed
the
feat/swift-wasm-support-v2
branch
from
aba49b8 to
312bd20
Compare
3 participants
Summary
This PR adds support for compiling sqlite-nio to wasm using the Swift SDK for WebAssembly.
This PR is part of a larger effort by a company called PassiveLogic to enable broad support for Swift WebAssembly.
Details
import NIOPosixandimport NIOCoreimports that introduced non-required dependencies onNIOPosixSQLITE_INTEGERnot working correctly on wasm builds. It reverted to a 32 bit integer, since wasm builds are currently 32 bit. Set up a newSQLiteDataIntegerTypethat handles the integer typing from a centralized definition.Testing done
swift build --swift-sdk swift-6.2.2-RELEASE_wasm --target SQLiteNIOcompletes without errors. Latest official Swift release.swift build --swift-sdk swift-6.3-DEVELOPMENT-SNAPSHOT-2025-12-07-a_wasm --target SQLiteNIOcompletes without errors. This is the latest daily snapshot of the swift toolchain.Usage notes
The current Swift 6.2.x breaks certain wasm executors that make testing wasm executables like this impossible. To fully test this running in the browser, recommend using Swift 6.3.x or later.
Impact Risk
There is a chance the changes to the manifest and exports could be a breaking change somewhere, if downstream dependencies relied on those pieces for their one and only intrinsic inclusion of the entire NIO dependency beyond NIOCore.
There are solutions to narrow the scope to just wasm compilation for those changes, if that is an issue. But I chose to leave them this way since it is a little cleaner solution.