Building SDKs in five languages

Adventures in ease of use for a modern REST API

My team just finished an awesome fintech REST API, but by itself that’s not enough. People aren’t going to start using it unless they have an easy way to get started. Let’s build five onramps, one for each of the most popular business programming languages: Python, Java, JavaScript, C#, and Ruby.

Build an onramp that’s easy for your customers to use (TEAM resources)

We have a few goals in mind for our SDKs:

• We want our code to look and feel like it was written by a developer familiar with best practices for each language. If you’re not a daily user of a particular programming language, it’s easy to put together code using patterns that were popular years ago.
• A developer using our SDK should be able to use autocomplete to discover the fields and methods they need, and we should support docblocks wherever possible.
• We want our code to be published on the package repositories for each programming language so that developers can integrate it into their development pipeline and discover updates automatically.

Read on to hear how we accomplished this for the Lockstep SDKs.

TypeScript and JavaScript software developer kits

We chose TypeScript for our first SDK because one of our developers had already hand-written code to use our API for one of our TypeScript projects. We adapted some of the code and set up docblocks so that our library could provide rich autocomplete.

A few issues we faced during the TypeScript SDK generation process:

• TypeScript supports docblocks that include embedded markdown. We were able to include links in our documentation that go to our developer documentation site to make the popup help more useful.

Markdown style comments in documentation allowed hyperlinking to detailed tutorials

• We chose to group all of our API functions by category. This reduces the size of the autocomplete popup and it prevents a user from accidentally autocompleting createCompanies() when they intended to call createContacts().

Autocomplete worked better with a shorter list of client objects

• TypeScript supports the notion of non-nullable objects. This can help remove all the common “test-for-null” code that tends to litter Java-like languages. It’s worth noting that Swagger uses the required: true tag to indicate whether query parameters are nullable, but when describing a data schema, it uses nullable: true instead.
• In TypeScript as well as C#, parameters that are required should generally appear first before parameters that are optional. This means you’ll be able to omit some parameters to a function call to make your code look cleaner.
• Since TypeScript supports asynchronous programming, we made all the methods fully async/await. We chose to use the axios library for HTTP calls; but we may choose to replace that with NodeJS’ built-in https module to reduce dependencies.
• One of the less obvious decisions was the return type. We chose to have all of our API calls return a LockstepResponse<T> object. This allows us to return a nice clean “success” value, and so that a programmer using the library would only have to write one code path that can observe either the valid result data or the error object as they prefer.

After all that work was done, we published our TypeScript SDK on NPMJS. During testing, we that it didn’t work correctly when added to a JavaScript project! As it turns out, we hadn’t configured our NPM build process to compile TypeScript into JavaScript before we built our package. With a bit of help from a good tutorial we were able to publish a working package.

We then discovered that that wasn’t enough. Some users could download our package and start working with it directly, but others would get no types, no classes, nothing. The root cause was that we were bundling too many files into the repository. A better solution was to implement RollupJS, a program that consolidated our package into three files:

• A single unified JavaScript file;
• A single “.d.ts” file, containing type information for TypeScript; and
• A “.map” file that contained sourcemaps for debugging.

With this change, our SDK now imports correctly into either JavaScript or TypeScript projects, and autocomplete shows rich documentation in VS Code:

Example autocomplete popup in VS Code

DotNet Core software developer kit

Since our typical daily programming language is C#, we had access to lots of developers to guide the design of our C# DotNet Core SDK. We chose the most current version of DotNet, version 6, due to the performance improvements it offers. Depending on requests from our clients we may add support for other DotNet versions in the future.

• C# uses XMLDOC, an older documentation block format. Each XML docblock is identified by triple-slash comments above each function and variable. Fortunately, most IDEs now recognize markdown comments so you can use links between each documentation block.
• Most C# projects use four-digit version numbers. Since so many other languages use three numbers for semver, we chose to just leave our fourth number as zero. We decided to use the format (year).(week).(build) so that we could easily identify what software or SDK a customer uses.
• C# is a strongly typed language, and our API uses PATCH for object updates. We chose PATCH because we wanted users to be able to pass in only a list of changed fields, rather than PUT, which typically replaces the object with a new one. We used dynamic object in C# for the body of our PATCH calls; however we are considering whether to replace this with a list of name-value pairs in the future so that our API calls can be more strongly typed.

After we completed our C# SDK, we noticed something odd: Certain parameters would have no documentation! After a bit of investigation, this turned out to be caused by a quirk of Swagger. We had to turn on a feature called “Render Objects With allOf” — the reasons are tricky to explain but this helped make sure that all of our objects had the correct documentation.

The Java software development kit

Next, we chose to tackle Java — and this one was a doozy. It’s been about a dozen years since I worked in Java regularly, and the language has only become stricter over time.

• Java is strictly a one-class-per-file language. This exploded the size of our API, at least when you count the number of files or the number of import statements per file.
• One interesting error is that Java doc blocks expect type references in docblocks to be in the form {@link classname} — Our Java IDEs complained a lot about this one!
• Java does support generic classes, but the support is not quite as flexible as that offered by TypeScript or by C#/DotNet. We solved this by using a RestRequest<T> class, which encapsulates each API call so that the code can be shared regardless of the result data.

An example of a generic RestRequest<> call in Java

• Although Java does support asynchronous programming, we ran out of time with our first release, and it isn’t yet complete. We’re hoping to implement this across the board soon.
• Opening our Java project in Visual Studio Code helped to catch lots of interesting issue reports. We scrolled through the project and identified all the compiler warnings and used this to help clean up unfamiliar areas.
• Java requires getters and setters to handle member fields within a class. These can be very tedious to write, but we’re using a code generator, so our work is a breeze.
• I didn’t find a native JSON parser or a native HTTP client in Java, so I chose to use Google’s GSON library and Apache’s HTTP components. Don’t most languages have these features built-in by now?
• Once we began testing the Java SDK, we started to get lots of exceptions. As it turned out, Java’s native date-time parsing couldn’t reliably handle the types of dates we chose to use in our API. We ended up rendering all dates as strings to avoid exceptions, although we’ll keep looking for a better solution in the future.

An unexpected challenge was the structure of our SDK project. I had completely forgotten that Java requires all classes to be named according to the owner’s domain-name. This meant that rather than just creating a folder called src, I had to create src/main/java/io/lockstep/api — took me a while to remember that quirk.

When the time came to publish our Java SDK, we found an interesting challenge: We couldn’t find a way to publish our JAR file manually. Both NuGet and NPMJS provided easy walkthroughs, but Maven was more difficult. We chose to spend the time to implement GitHub workflows to build, test, and publish our Maven library.

The GitHub workflow approach was so successful that we chose to implement GitHub workflows for all our other SDKs afterwards. Special thanks to the the Lockstep Devops team for solving that challenge — it’s a relief to know that our SDKs automatically build and publish after each commit.

The Python software development kit

Our next challenge was Python. Most of us write Python code from time to time — it is the world’s most popular development language after all — but few of us have built PyPi packages and published them to the world.

• Our first challenge was Python docstrings. These are … not well standardized! One of our interns helped me research both Google’s Python Style Guide and the Sphinx docstring format. In addition, we found many historical comments from Python’s early days suggesting other alternative forms of docstrings. We chose to use the Google style guide, which looks like this:

Python docstring example using Google’s style guide

• Remember to put your docstrings within each class and method, rather than above them! Python expects docstrings to be the first statement within a class or method. Once working, rich documentation appears in Python like this:

• We created an __init__.py class that added import statements for all of our objects so that each class didn’t need its own import statements.
• Python is natively an untyped language, where every variable is assumed to be a dynamic object. We wanted our SDK to provide helpful guidance, so we chose to add type annotations wherever possible.
• We decided to create our data models as Python dataclass objects, which provided a nice balance of usability and simplicity. I haven’t found a good way to add docstrings to member variables yet, but I hope to solve that someday.

Python dataclass example showing documentation at the class level but not at the field level

• We ended up using snake_case for our Python method names and PascalCase for our Python class names. During our experimentation phase, lots of developers weighed in on their preferred naming standards and we ended up changing this back and forth quite a few times!

The Python SDK was fun to create: some Python style guides insisted that files be no more than 80 characters in width, which required some clever updates to our code generator to reformat our docblocks. This code was fun to write and featured some clever logic to ensure that paragraph breaks and bullet point lists were preserved correctly in markdown.

After we completed the parsing logic, we discovered that our docblocks looked terrible! Many of our markdown URLs would be fifty or sixty characters in length by themselves, which often caused them to wrap onto a line all by themselves. The magic solution was to avoid wrapping the line until it reached the 40 character mark — this prevented us from having a single word on its own line in the markdown.

The Ruby software development kit

Of the languages we chose to implement, I’m least familiar with Ruby. Ruby is an untyped language, meaning every variable is a dynamic object. Its syntax is very different than if you’re familiar with C-derived languages.

Fortunately, our team works with developers in India who are well-versed in the language, and they helped guide the Ruby SDK to launch.

• In Ruby, the word include is a reserved keyword. Some of our APIs had query string parameters named “include”, so we had to create a converter to rename it.
• Our Ruby SDK uses snake_case for all methods and filenames, but we chose to use PascalCase for our module and class names.
• Ruby allows you to use named parameters, but it doesn’t allow you to specify types. If you use Ruby, your IDE can’t tell you if your code passes the correct data to a function — but we can try to provide hints in the documentation blocks.

A Ruby function call with named parameters; the types are hints in the documentation

• Although Ruby is by nature untyped, it does provide a helpful annotation called attr_accessor that allows you to specify named variables within a class. We were able to add docblocks in Rdoc format for each variable.

Examples of Ruby attr_accessor named variables

The Ruby SDK took the most time to complete, but the end result seems encouraging.

The journey to launching five SDKs was a fun experiment in code generators, GitHub actions, and reading tons of documentation and recommendations from developers of all kinds.

This exercise gave me a lot of appreciation for strictness in a programming language. Languages with strict standards are hard at the beginning, but once you understand them you can produce reliable results and the IDE helps you along the way. Untyped languages, or languages with fewer standards, may be easier at the start — but you make more mistakes in the long run.

One choice we made after the project started was to add SdkName and SdkVersion headers to all of our API calls so that we can track usage. We’re building dashboards to keep track of how well our partners and customers like these software development kits. We hope you’ll like them too!

Ted Spence teaches at Bellevue College and leads engineering at Lockstep. If you’re interested in software engineering and business analysis, I’d love to hear from you on LinkedIn.