Handling custom logic in API Gateway

Background

Imagine you have an API backend that uses a proprietary auth mechanism but you would like to expose the API publicly on the API gateway with a standard auth mechanism like OAuth 2.0. This would require some custom auth translation logic at the API gateway layer which does the following processing steps:

1. Retrieve the OAuth2.0 access_token from incoming request and exchange it for a token corresponding token for the backend service. This step may involve fetching the token from a cache if it exists and otherwise calling some custom library/code to get a translated token and cache the result.

2. Call the backend service using the exchanged token. If the backend responds with a token expired error, then go back to step 1 and after getting a new token, repeat step 2. Once the backend service responds with a success or any other error, respond back to the caller.

Points to note: Most commercial API gateways have policies configurable for standard security, logging, and translation requirements. When more customized business logic processing is needed, usually the OOTB policies don’t help and there is a need to write and host our custom code.

Problem Statement

Should we put business logic requiring custom code in API gateway layer?

In other words when should we make service callout to custom logic from gateway proxy or push the custom logic closer to the backend service? Let’s take a closer look and compare.

Approaches

Option 1 - Implement the custom logic as a service and call out to it from the gateway. Also take care or orchestrating multiple such callouts (when needed) as proxy configurations in gateway.

Option 2 - Separate out all custom logic to a separate layer which acts as an inner gateway for the upstream service call.

Comparison

Unless the API gateway allows custom code based policies, the custom code has to be implemented and hosted outside of the gateway runtime. That makes it one more runtime to manage and pay for. Communication with this custom code has to happen using another API but is still an over the wire call even if it is highly optimized as in the case of gRPC. The custom code may be co-located to the gateway runtime in another pod if K8S is in use or could be running as a serverless function or in an app container.

One can argue that doing an externally hosted custom code or a code based inner gateway, both involve another runtime and an additional over the wire call. And that is true. But there are some subtle differences.

• Implementing the custom logic requires creating another API which can respond based on the execution of the custom logic. In our auth translation example, this would be an API which can accept an input token and return a translated token (for success case) and error message if not. Doing all this is additional work and cognitive overhead. Further the gateway proxy needs to have configurations to make the callout to this API and handle the success/failure scenarios. That adds complexity. Whereas when the custom code goes inside the inner gateway, no new API is created. The outer gateway calls out to it exactly the same way it calls the backend service.

• As long as there is only one piece of custom logic, the comparison might still look on par. But things change once you have more such custom pieces. For ex. let’s say in addition to the auth transation, there is also an additional custom rate-limiting which does rate-limiting based on the geo from which the request is originating. For Option 1 this will mean one more API introduced, one more over the wire callout and handling one more response. Whereas in case of Option 2 it is adding one more middleware to the request processing chain which happens in same process context so no new network calls. The latency, complexity and cognitive overhead quickly adds up as more custom policies get created in the case of Option 1.

• From troubleshooting and performance monitoring perspective it is helpful if logs and metrics from custom policies are correlated and colocated. This is easier to achieve when all custom policies are also colocated and executed from a common context which is the case with Option 2. With Option 1 it can sometimes be a challenge to get to a proper understanding of what happened in the custom code and what happened in gateway when it processed the callout response to make further decisions. If/then/else processing done in gateway configurations as xml for example make it harder to instrument.

Additional Thoughts

• For performance, manageability and governance reasons it is a good idea to not overload API gateway with complex custom logic when it doesn’t have first class support for embedding and running such custom code.

• It’s best to use the API gateway for capabilities it is optimized for and which come OOTB. A good rule of thumb is to do configurations only in the gateway for policies which execute natively on the gateway runtime.

• When complex custom policies are required it is better to use an API gateway which allows custom code natively. Such a gateway can be deployed as an inner gateway which works in conjunction with an outer gateway doing standard (non-custom) policies with configurations.

• If the inner gateway also has all standard policies supported OOTB, we can do away with the outer gateway. However having two levels has its benefits too in terms of flexibility and broader set of capabilities.

• Putting callouts to multiple services in gateway makes it more of a service composition/integration platform than an API gateway which it was probably not meant, designed and optimized for.