Soto Version 7 Alpha

An alpha for Soto version 7.0 has just been released. This version has some API improvements and considerable internal changes. Below is listed some of the more major changes and how they affect the library and its users.

Swift Concurrency

The library is now completely written using Swift concurrency. The EventLoopFuture based internals are gone along with the EventLoopFuture APIs. This allows us to make full use of all the features of Swift concurrency and also use the new Swift concurrency based APIs from AsyncHTTPClient which have better support for streamed response payloads.

Streamed Payloads

Operations that returned streamed responses, instead of returning the payload by repeated calling a closure now return the streamed payload as an AWSHTTPBody which conforms to a AsyncSequence of ByteBuffers. You can extract the payload using the pattern below or alternatively use the function AWSHTTPBody.collect(upTo:) to collate the stream of buffers into one.

let result = try await S3.getObject(.init(bucket: "MyBucket", key: "name"))
for try await buffer in result.body {
    processBuffer(buffer)
}

Event streams

Event stream APIs have previous been difficult to implement. S3.SelectObjectContent had a custom implementation, but other operations that received an event stream did not work. The move to AsyncSequence based payloads has simplified implementing these and now there is a generic solution for them all. This solution has been tested on S3.SelectObjectContent and Lambda.InvokeWithResponseStream. The stream of events are accessed from an AWSEventStream which conforms to AsyncSequence. At this point in time there is no support for sending event streams.

Request encoding/Response decoding

The solution for encoding of request header, query values etc and decoding of response header values has always been unsatisfactory. When encoding a request's headers Soto would use Mirror to extract the values out of the input shape. When decoding response headers the header values were added to the input to the Codable decoders, which meant the input had to be in a form the values could be added. JSON reponses had to be decoded to a dictionary before header values were added to the dictionary.

Both of these have been improved by passing a container to the encode/decode functions (via userInfo) which holds the request/response details. The encode functions use their request container to update the request headers, query parameters. The decode functions use their response container to extract header values.

These changes give us performance improvements at both the encoding and decoding stages. We aren't using Mirror while encoding, instead we are writing the values directly to the headers/query arrays during the encode(to:) functions. Decoding of JSON files doesn't require us use our custom DictionaryDecoder. Instead we can use the most up to date JSONDecoder which is considerably faster.

Middleware

Previously Soto middleware provided you with a chance to edit the request before it was sent or the response just after receiving it. This has changed structure to allow more flexibility. You are provided with the request, and a closure to call the next middleware and are expected to return either the response from the closure or an edited version of it. Below is an example middleware.

struct MyMiddleware: AWSMiddlewareProtocol {
    func handle(_ request: AWSHTTPRequest, context: AWSMiddlewareContext, next: AWSMiddlewareNextHandler) async throws -> AWSHTTPResponse {
        let request = processRequest(request)
        let response = try await next(request, context)
        return processResponse(response)
    }
}

This middleware is only editing requests and responses but there is nothing stopping you doing something more complex. In actual fact the middleware is flexible enough now that every part of the Soto request/response processing (signing, error handling, retries, endpoint discovery) is now done with middleware.

Middleware Stack

We also use a result builder to build the middleware stack. When you provide middleware to AWSClient or your service it can be either as an individual middleware or a stack of middleware.

let client = AWSClient(
    middleware: AWSMiddlewareStack {
        AWSLoggingMiddleware()
        MyMiddleware()
    }
)

SotoCrypto removed

The library now has platform requirements of macOS v10.15 and iOS v13. Because of this we can now use the swift-crypto package on all platforms and can get rid of the platform check in the Package.swift. This will fix the issues when cross compiling for Linux on macOS where it would not include swift-crypto. Also it means there is one less package to maintain.