Architecture for isomorphic API Client with TypeScript

By Toru Kobayashi (@koba04)

This article will introduce how we've built @kintone/rest-api-client and its architecture.

This article is written based on @kintone/rest-api-client@1.8.2.

What is @kintone/rest-api-client?

@kintone/rest-api-client (rest-api-client) is a REST API client for Kintone and is one of the packages in the kintone/js-sdk repository. kintone/js-sdk manages multiple packages as a Monorepo repository. If you are interested in how we use Monorepo to manage multiple packages, please visit the following article I wrote before.

blog.kintone.io

rest-api-client is an isomorphic API client, which means it works on both browsers and Node.js environments.

The overall structure

The project structure of rest-api-client is like the following.

➜  tree -L 2 packages/rest-api-client/src
packages/rest-api-client/src
├── KintoneFields
│   ├── exportTypes
│   └── types
├── KintoneRequestConfigBuilder.ts
├── KintoneResponseHandler.ts
├── KintoneRestAPIClient.ts
├── __tests__
│   ├── KintoneRequestConfigBuilder.test.ts
│   ├── KintoneResponseHandler.test.ts
│   ├── KintoneRestAPIClient.test.ts
│   ├── setup.ts
│   └── url.test.ts
├── client
│   ├── AppClient.ts
│   ├── BulkRequestClient.ts
│   ├── FileClient.ts
│   ├── RecordClient.ts
│   ├── __tests__
│   └── types
├── error
│   ├── KintoneAbortSearchError.ts
│   ├── KintoneAllRecordsError.ts
│   ├── KintoneRestAPIError.ts
│   └── __tests__
├── http
│   ├── AxiosClient.ts
│   ├── HttpClientInterface.ts
│   ├── MockClient.ts
│   └── index.ts
├── index.browser.ts
├── index.ts
├── platform
│   ├── UnsupportedPlatformError.ts
│   ├── __tests__
│   ├── browser.ts
│   ├── index.ts
│   └── node.ts
└── url.ts

In the following paragraphs, I'll explain the client/, http/, and platform/ directories.

Controlling dependencies

As I mentioned, rest-api-client is intended for browser and Node.js environments. We've designed it to support both by separating processes according to environment.

HTTP Client

rest-api-client uses axios as its HTTP client. Since Axios alone supports both browsers and Node.js environments, it's possible for us to build a client that works for both without any abstraction. But we have defined an abstracting interface through which to use the HTTP client.

The following is the HttpClientInterface.

import FormData from "form-data";
export interface HttpClient {
  get: <T extends object>(path: string, params: object) => Promise<T>;
  getData: (path: string, params: object) => Promise<ArrayBuffer>;
  post: <T extends object>(path: string, params: object) => Promise<T>;
  postData: <T extends object>(path: string, params: FormData) => Promise<T>;
  put: <T extends object>(path: string, params: object) => Promise<T>;
  delete: <T extends object>(path: string, params: object) => Promise<T>;
}

export type ErrorResponse<T = any> = {
  data: T;
  status: number;
  statusText: string;
  headers: any;
};

export type Response<T = any> = {
  data: T;
  headers: any;
};

export type HttpMethod = "get" | "post" | "put" | "delete";
export type Params = { [key: string]: unknown };

export type ProxyConfig = {
  host: string;
  port: number;
  auth?: {
    username: string;
    password: string;
  };
};

export interface HttpClientError<T = ErrorResponse> extends Error {
  response?: T;
}
export interface ResponseHandler {
  handle: <T = any>(response: Promise<Response<T>>) => Promise<T>;
}

export type RequestConfig = {
  method: HttpMethod;
  url: string;
  headers: any;
  httpsAgent?: any;
  data?: any;
  proxy?: ProxyConfig;
};

export interface RequestConfigBuilder {
  build: (
    method: HttpMethod,
    path: string,
    params: Params | FormData,
    options?: { responseType: "arraybuffer" }
  ) => Promise<RequestConfig>;
}

rest-api-client hides the implementation details of HttpClientInterface from the outside of http/ directory by using the HTTP client through the interface. We have AxiosClient, the interface implemented with axios. Modules using the HTTP client don't know that it internally depends on axios.

// implement HttpClient Interface
export class AxiosClient implements HttpClient {
  private responseHandler: ResponseHandler;
  private requestConfigBuilder: RequestConfigBuilder;

  constructor({
    responseHandler,
    requestConfigBuilder,
  }: {
    responseHandler: ResponseHandler;
    requestConfigBuilder: RequestConfigBuilder;
  }) {
    this.responseHandler = responseHandler;
    this.requestConfigBuilder = requestConfigBuilder;
  }

  public async get(path: string, params: any) {
    // build a request config to use kintone REST API
    const requestConfig = await this.requestConfigBuilder.build(
      "get",
      path,
      params
    );
    return this.sendRequest(requestConfig);
  }

  public async post(path: string, params: any) {
    // build a request config to use kintone REST API
    const requestConfig = await this.requestConfigBuilder.build(
      "post",
      path,
      params
    );
    return this.sendRequest(requestConfig);
  }
  
  // 省略

  private sendRequest(requestConfig: RequestConfig) {
    // pass a Promise returned from `Axios` to a response handler received from a constructor
    return this.responseHandler.handle(
      Axios({
        ...requestConfig,

        // NOTE: For defining the max size of the http request content, `maxBodyLength` will be used after version 0.20.0.
        // `maxContentLength` will be still needed for defining the max size of the http response content.
        // ref: https://github.com/axios/axios/pull/2781/files
        // maxBodyLength: Infinity,

        maxContentLength: Infinity,
      })
    );
  }
}

We've also defined interfaces with HTTP requests and responses, not only with the HTTP client. These abstracting interfaces will be helpful if we want to migrate from axios to another HTTP client library, because they enable us to encapsulate a migration process in the HTTP layer.

In addition, we've provided the following interfaces.

HttpClientError ... an interface for an HTTP error
RequestConfig, RequestConfigBuilder ... an interface for an HTTP request
ResponseHandler ... an interface for an HTTP response

Why are RequestConfigBuilder and ResponseHandler necessary?

RequestConfigBuilder and ResponseHandler are modules to process requests and responses according to some logic specific to the domain of Kintone. For example, RequestConfigBuilder converts a GET request into POST if the requested URL is longer than the limit defined by Kintone. We don't want to add the logic into the HTTP layer, so we've decided to pass the logic from outside of the HTTP layer as RequestConfigBuilder.

github.com

KintoneResponseHandler handles the responses from the REST API of Kintone.

github.com

Currently, we don't have any plans to migrate the HTTP client library from axios because it has useful features like proxy and client certificate authentication support. But in the future, we may want to use different HTTP client libraries for different environments, so we think this abstraction is important to make the migration easy.

We also have MockClient as implementation of HttpClientInterface. MockClient is an HTTP client for unit testing. In addition to implementation for the HTTP client interface, the client has features like returning arbitrary responses and recording requests and responses. This allows us to easily write unit tests using the HTTP client without any mock libraries.

  let mockClient: MockClient;
  let recordClient: RecordClient;
  beforeEach(() => {
    const requestConfigBuilder = new KintoneRequestConfigBuilder({
      baseUrl: "https://example.cybozu.com",
      auth: { type: "apiToken", apiToken: "foo" },
    });
    // create a MockClient and use it for the HTTP client
    mockClient = buildMockClient(requestConfigBuilder);
    const bulkRequestClient = new BulkRequestClient(mockClient);
    recordClient = new RecordClient(mockClient, bulkRequestClient);
  });
  describe("addRecords", () => {
    const params = { app: APP_ID, records: [record] };
    const mockResponse = {
      ids: ["10", "20", "30"],
      revisions: ["1", "2", "3"],
    };
    let response: any;
    beforeEach(async () => {
      // set a mock response
      mockClient.mockResponse(mockResponse);
      response = await recordClient.addRecords(params);
    });
    it("should pass the path to the http client", () => {
      expect(mockClient.getLogs()[0].path).toBe("/k/v1/records.json");
    });
    it("should send a post request", () => {
      expect(mockClient.getLogs()[0].method).toBe("post");
    });
    it("should pass app and records to the http client", () => {
      expect(mockClient.getLogs()[0].params).toEqual(params);
    });
    it("should return a response having ids, revisions, and records", () => {
      expect(response).toEqual({
        ...mockResponse,
        records: [
          { id: "10", revision: "1" },
          { id: "20", revision: "2" },
          { id: "30", revision: "3" },
        ],
      });
    });
  });

Dependencis for each environment

In addition to the HTTP layer, there are differences between browsers and Node.js environments like authentication methods. platform/ is the directory to handle it.

We've defined the following interface to abstract the platform layer.

type PlatformDeps = {
  readFileFromPath: (
    filePath: string
  ) => Promise<{ name: string; data: unknown }>;
  getRequestToken: () => Promise<string>;
  getDefaultAuth: () => DiscriminatedAuth;
  buildPlatformDependentConfig: (params: object) => object;
  buildHeaders: () => Record<string, string>;
  buildFormDataValue: (data: unknown) => unknown;
  buildBaseUrl: (baseUrl?: string) => string;
  getVersion: () => string;
};

github.com

Then, we've implemented the interface for both environments. We throw UnsupportedPlatformError if a feature is not supported in the target environment.

platform/node.ts

export const readFileFromPath = async (filePath: string) => {
  const data = await readFile(filePath);
  const name = basename(filePath);
  return { data, name };
};

export const getRequestToken = () => {
  // This is only supported on a browser environment
  throw new UnsupportedPlatformError("Node.js");
};

export const getDefaultAuth = () => {
  // This is only supported on a browser environment
  throw new UnsupportedPlatformError("Node.js");
};

export const buildPlatformDependentConfig = (params: {
  clientCertAuth?:
    | {
        pfx: Buffer;
        password: string;
      }
    | {
        pfxFilePath: string;
        password: string;
      };
}) => {
  const clientCertAuth = params.clientCertAuth;

  if (clientCertAuth) {
    const pfx =
      "pfx" in clientCertAuth
        ? clientCertAuth.pfx
        : fs.readFileSync(clientCertAuth.pfxFilePath);
    const httpsAgent = new https.Agent({
      pfx,
      passphrase: clientCertAuth.password,
    });
    return { httpsAgent };
  }
  return {};
};

export const buildHeaders = () => {
  return {
    "User-Agent": `Node.js/${process.version}(${os.type()}) ${
      packageJson.name
    }@${packageJson.version}`,
  };
};

export const buildFormDataValue = (data: unknown) => {
  return data;
};

export const buildBaseUrl = (baseUrl: string | undefined) => {
  if (typeof baseUrl === "undefined") {
    throw new Error("in Node.js environment, baseUrl is required");
  }
  return baseUrl;
};

export const getVersion = () => {
  return packageJson.version;
};

How do we switch the platform dependencies?

We do not switch the dependencies at runtime. If we did it, module bundlers like webpack or rollup would bundle dependencies for the Node.js environment that browsers would never use. rest-api-client provides UMD builds built with rollup, so we wouldn't like them to include dependencies for the Node.js environment.

To avoid it, we've made separate entry files and injected environmental dependencies into each of them.

The entry file for browsers: src/index.browser.ts

import { injectPlatformDeps } from "./platform/";
import * as browserDeps from "./platform/browser";

injectPlatformDeps(browserDeps);

export { KintoneRestAPIClient } from "./KintoneRestAPIClient";

The entry file for Node.js: src/index.ts

import { injectPlatformDeps } from "./platform/";
import * as nodeDeps from "./platform/node";

injectPlatformDeps(nodeDeps);

export { KintoneRestAPIClient } from "./KintoneRestAPIClient";
export * as KintoneRecordField from "./KintoneFields/exportTypes/field";
export * as KintoneFormLayout from "./KintoneFields/exportTypes/layout";
export * as KintoneFormFieldProperty from "./KintoneFields/exportTypes/property";

We've used the platform dependencies injected in the entry files like this.

export class KintoneRestAPIClient {
  record: RecordClient;
  app: AppClient;
  file: FileClient;
  private bulkRequest_: BulkRequestClient;
  private baseUrl?: string;

  constructor(options: Options = {}) {
    this.baseUrl = platformDeps.buildBaseUrl(options.baseUrl);

    // ...

github.com

This allows us to easily write environment-dependent tests by replacing platform-specific implementation in test files.

github.com

Client

Kintone has many REST API endpoints, so we've built multiple categorized clients in rest-api-client. These clients receive an HTTP client as a parameter of its constructor. In unit testing, we pass them MockClient, which allows us to test them without dealing with actual API endpoints.

Currently, we have to prepare MockClient in each test case, and it is a bit annoying. We'd like to provide a more maintainable way to manage our mock data.

Other Tips

Rolling out a new feature behind feature flags

Breaking changes are always a burden on developers who use our libraries for their products, even if we upgrade a major version for such breaking changes. We think we should ensure that any update path we provide to developers is as gradual as possible. For this reason, we've added feature flags to rest-api-client.

ref. github.com

It allows us to take a strategy to release a new feature behind a feature flag that is off by default, which means that developers can try the new feature as an opt-in. Later, we can turn the feature flag on by default and release it as a major release. At the time, we don't remove the feature flag so that developers can disable the feature as an opt-out.

We anticipate that we cannot apply the feature flag strategy to some breaking changes. Still, by providing better warning messages for example, we will always do our best to avoid breaking products that uses rest-api-client.

Start implementing To-dos

rest-api-client uses Jest as a unit testing framework. Jest provides a way to write test cases as to-do's like it.todo("should be bar"); before starting to implement an actual unit test. In rest-api-client, we utilize this feature to separate the development process into two phases, the phase of thinking about specifications and the phase of implementing those specifications. This separates the implementation process into two phases, thinking the specification and the actual implementation based on the specification. As a result, we can focus on the specification and the actual implementation independently.

We also use test.each to write tests for complicated pure functions. It allows us to write tests where it is easy to recognize inputs and expected results of the function.

tests/url.test.ts

test.each`
  endpointName | guestSpaceId | preview      | expected
  ${"record"}  | ${undefined} | ${undefined} | ${"/k/v1/record.json"}
  ${"record"}  | ${undefined} | ${false}     | ${"/k/v1/record.json"}
  ${"record"}  | ${undefined} | ${true}      | ${"/k/v1/preview/record.json"}
  ${"record"}  | ${3}         | ${undefined} | ${"/k/guest/3/v1/record.json"}
  ${"record"}  | ${3}         | ${false}     | ${"/k/guest/3/v1/record.json"}
  ${"record"}  | ${3}         | ${true}      | ${"/k/guest/3/v1/preview/record.json"}
  ${"record"}  | ${"3"}       | ${undefined} | ${"/k/guest/3/v1/record.json"}
  ${"record"}  | ${"3"}       | ${false}     | ${"/k/guest/3/v1/record.json"}
  ${"record"}  | ${"3"}       | ${true}      | ${"/k/guest/3/v1/preview/record.json"}
`("buildPath", ({ endpointName, guestSpaceId, preview, expected }) => {
  expect(buildPath({ endpointName, guestSpaceId, preview })).toBe(expected);
});

Conclusion

In this article we've introduced how we've created rest-api-client. It's important for us to think of an architecture to write an isomorphic API client that runs on browsers and Node.js environments.

Kintone Engineering Blog

Learn about Kintone's engineering efforts. Kintone is provided by Cybozu Inc., a Tokyo-based public company founded in 1997.

Architecture for isomorphic API Client with TypeScript

What is @kintone/rest-api-client?

The overall structure

Controlling dependencies

HTTP Client

Dependencis for each environment

Client

Other Tips

Rolling out a new feature behind feature flags

Start implementing To-dos

Conclusion