Potassium

The code you run on GPUs
Potassium is an open-source HTTP serving framework built for AI, sponsored by Banana.
Quickstart:
Installation
The fastest way to get up and running is to use the Banana CLI, which downloads and runs your first model.
1.Install the CLI with pip
pip3 install banana-cli
2.Create a new project directory with
banana init my-app
cd my-app
3.Start the hot-reload dev server
banana dev
The hot reload feature of the dev server can save you hours of time and focus.

It recognizes when you change your potassium app handlers, and hotpatches those changes into the server while keeping your models hot in memory. No more waiting for 30s+ model reloads every time you make a code change.
4.Call your local API (from a separate terminal)
curl -X POST -H "Content-Type: application/json" -d '{"prompt": "Hello I am a [MASK] model."}' http://localhost:8000/
​🎉
 Congrats! You've ran a local inference on your Potassium backend, with a BERT model
Making it your own
​app.py is the core of Potassium, and where you'll be adding your custom code and models.
Read more about it and Potassium here:
app.py
The banana cli also provides these files:
5.​requirements.txt - python package management
6.​download.py - a script ran at build-time to download models
7.​Dockerfile - build steps, generally can be ignored unless you know what you're doing
Deploying
1.Push your Potassium app to Github with the include Dockerfile. It must be at the root of the repo, for now. (monorepo support coming soon)
2.Click "+ New MODEL" in the banana app, select your new repo, and deploy
Next, can then call your BERT hello world model from the SDKs, using the below json as model_inputs:
{
    'prompt': 'Hello World! I am a [MASK] machine learning model.'
}
And the SDK will return outputs such as:
{
    // some metadata
    ...
    // some metadata
    "modelOutputs": [
      [
        {
          "score": 0.0529061034321785,
          "token": 3722,
          "token_str": "simple",
          "sequence": "hello world! i am a simple machine learning model."
        },
        {
          "score": 0.050797536969184875,
          "token": 3143,
          "token_str": "complete",
          "sequence": "hello world! i am a complete machine learning model."
        },
      ]
    ]
}
Don't want to use Potassium? 
You can deploy any Docker workload onto Banana GPUs. Follow this guide to go beyond the Potassium framework and build your own Banana-compatible repo from scratch.
​
Core Concepts - Previous
Inference Server
app.py
Last modified 2mo ago