How it started

In 2023, OpenBB introduced the Terminal Pro, a web application that provides seamless access to financial data with powerful AI features and customization capabilities.

Before that, we already facilitated access to financial data through a command-line tool (OpenBB Terminal) and a Python library (OpenBB SDK).

However, the team identified the need to create a modern user interface, providing a more intuitive experience and reducing the learning curve for new users.

And that was the beginning of the OpenBB Terminal Pro.

The OpenBB Platform

During the development phase of the Terminal Pro, we recognized that the pivotal feature for that transition was to create an API capable of routing all frontend requests to various data providers.

So, we did that by leveraging well-known Python packages such as FastAPI and Pydantic, which established the foundation for what we now call openbb-core.

The openbb-core is a package written in Python that encompasses most of the logic to run a REST API whose endpoints return financial data from several providers.

When the API receives incoming requests, it performs the following actions:

1. Identify the provider to send the query
2. Send a request to the provider
3. Return the data inside a well-defined model

We refer to the assembly of providers (routers), and their interaction with the openbb-core, as the OpenBB Platform.

Backend

Routers

As the diagram above shows, there is a layer between openbb-core and the ‘Providers’ where ‘Routers’ such as openbb-equity or openbb-economy live. That’s where endpoints are defined.

For example, the openbb-equity router contains the endpoint equity/profile to get general data about a company.

But it also contains equity/price/historical (which belongs to the sub-router equity/price) to get historical price data.

The Platform spawns multiple nested routers, which connect to over 100 endpoints. These routers establish communication with providers to execute data queries.

The API documentation in the screenshot below is the result of assembling all of the components of this structure.

Providers

The data providers orbit in the outermost layer of our code base. These are the ones actually grabbing the data from the external sources and returning it to the router.

To achieve this, the implementation follows a TET (Transform-Extract-Transform) pipeline pattern. Take a look at the following example to get data from Benzinga.

1. Transform: when the query arrives, the first step is to check if the parameters are valid and apply defaults.

2. Extract: the provider accesses the external data source, usually a public/private API, but it can also be local files, a database, or any available source.

3. Transform: validate the structure and apply additional transformations to the raw data, such as standardizing the date format or dropping specific fields.

When the 3 steps are complete, the data is returned to the router and flows back to the application that made the request.

Frontend

One of our internal requirements was to keep providing a Python library with a user-friendly interface that resonates with Python developers, quants, and data scientists.

We faced some challenges during this phase, but the key insight was to treat this library as any other client of the API, mirroring the approach taken with the development of Terminal Pro or the Excel Add-in.

Both Terminal Pro and Excel act as frontends that display the data coming from openbb-core (see architecture diagram above).

Since the API is very well documented and follows the same structure for all endpoints, we leverage the OpenAPI specification to generate our client applications' code.

This helps reduce maintenance and ensure consistency across all products.

To make the Python library available, we follow a similar approach, where a script fully traverses the API structure and dynamically generates Python functions that send the same queries to openbb-core.

Different client, same data

Here's a practical example illustrating the duality between the pure Python user and a web app sending HTTP requests to the platform:

1 - Install OpenBB with pip install openbb and make requests to the API:

a. Launch the server:

uvicorn openbb_core.api.rest_api:app

b. Send an HTTP request to the server (endpoint):

curl -X 'GET' \
  'http://localhost:8000/api/v1/equity/price/historical?provider=intrinio&symbol=AAPL' \
  -H 'accept: application/json'

2 - Jupyter notebook

The data obtained from these requests will be identical.

This behavior also extends to the Add-in for Excel where you can run the function with similar results:

The same goes for OpenBB Terminal Pro, where the data you see on this screenshot is served from the OpenBB Platform:

Conclusion

In summary, transitioning from a command-line tool to a web application motivated the complete decoupling of data acquisition (OpenBB Platform) from manipulation or visualization (Terminal Pro, Add-in for Excel,…).

This evolution has proven beneficial to the overall architecture of our products, enabling the creation of new user interfaces effortlessly on the same underlying data.

At the same time, we can now easily extend the core with new data providers or routers, which will be available to all the client applications.

Wondering how we implemented the extension architecture? Well, that's a topic for another blog post in the future, so stay tuned.

The code for data providers and core is open-source and available on GitHub. You can leverage it to create your own applications on top of the existing data providers or even integrate a new provider.

Check out the OpenBB Platform Documentation to know everything you need to start!