Getting Started in a Few Minutes

After this guide you'll have a simple BERT model in prod.

Setting up a dev environment

Install the Banana CLI with pip.

pip3 install banana-cli

Run the init command to create your "Potassium app"

banana init hello-world
cd hello-world

This downloads boilerplate and installs dependencies.

If you get an error such as "command not found: banana ", you probably haven't added the Python package bin location to your PATH variable. If that's the case, on Mac try something like: export PATH="/Users/<me>/Library/Python/3.9/bin:$PATH"

Your directory will now contain these files:

A freshly created potassium project runs a Huggingface BERT model, for sake of example.

Calling your project locally

Within your hello world potassium project, enter your virtual environment

. ./venv/bin/activate

... and start the dev server!

python3 app.py

You should see:

------
Starting server 🍌
Running init()
...
Serving on http://localhost:8000
------

Now open up a different terminal and hit the server with a simple cURL POST request

curl -X POST \
    -H "Content-Type: application/json" \
    -d '{"prompt": "Software developers start with a Hello, [MASK]! script."}' \
    http://localhost:8000/

Boom! 🎉 you just ran an inference on a BERT model on your local machine!

{
    "outputs": {
        "score": 0.8281897306442261,
        "sequence": "software developers start with a hello, world! script.",
        "token": 2088,
        "token_str": "world"
    }
}

Using the Banana SDK

cURL request are cool & you can continue to use them but as a convenience layer we'll use this opportunity to demonstrate the use of Banana's SDKs.

We'll be using the Python SDK for this example. You may find alternative SDKs here.

Although OK while testing, you should create a new directory for the following code, not to mix together your app's client and server files.

If you're still in the virtual environment from the first step in this guide, deactivate it

deactivate

Create a new directory

cd ../
mkdir banana-client
cd banana-client

Create a new Python virtual environment in that new directory

python3 -m venv venv
. ./venv/bin/activate

Install the Python SDK with pip

pip3 install banana-dev

The python client should be version 6.0.0 or above

Create a file called client.py with the following code:

client.py
from banana_dev import Client

# Create a reference to your model on Banana
my_client = Client(
    api_key="YOUR_API_KEY", # Found in app.banana.dev
    url="http://localhost:8000", # Pointed to your local instance of potassium
)

# Specify the model's input JSON
inputs = {
    "prompt": "Software developers start with a Hello, [MASK]! script",
}

# Call your model's inference endpoint on Banana
result, meta = my_client.call("/", inputs)
print(result)

This code is what will call your model in a production environment on Banana's GPUs.

For local testing, we set the url to http://localhost:8000 - this will be a different URL for your deployed project.

Lastly, we run the client code

python3 client.py

which prints the same as our cURL output from before:

{
    "outputs": {
        "score": 0.8281897306442261,
        "sequence": "software developers start with a hello, world! script.",
        "token": 2088,
        "token_str": "world"
    }
}

Awesome! Now it's time to deploy to Banana's serverless GPUs.

Deploying to Banana

It's time to get off localhost!

You have two options:

  1. deploy directly from the CLI, or

  2. connect Github so that future commits can be automatically deployed.

CLI Deployments

Run banana deploy from the root of your project. That's all. Really.

You'll need to be logged in to your Banana account in the browser for the CLI to authenticate you.

GitHub Deployments

Pushing to GitHub

You'll need a Github account to deploy, so set one up if you don't already have one.

Firstly, create a new repo in the Github UI. It may be private or public. Leave it empty.

From within your Potassium App directory, initialize a git project and push to remote

# Add recent changes
git add .
git commit -m "first changes"

# Set upstream to the Github Repo you created
git remote add origin https://github.com/USER_NAME/REPO_NAME.git
git branch -M main

# Push
git push -u origin main

Connect your account to banana

Log into Banana and follow the onboarding if you're a new user.

You'll be asked to connect your github, choose the repositories you want Banana to see

Go back to the Banana dashboard and click New Project -> Deploy from Github

Select the repo you want to deploy, and you can click into it and watch the build progress

Once built, the project will have "Deployed" status

Call it in prod!

We'll slightly modify our client.py file from before to call our deployed project.

  1. Find your model url by clicking into the project, and add it to client.py as https://<url-slug>.run.banana.dev

And fire off your first call!

python3 client.py

Once the call returns, the replica will remain warm a bit before shutting down.

Now go bananas!!

You've made a production call to a custom BERT model on Banana!

But this is just the beginning.

We encourage you to read Banana Usecases & Configuring Potassium, and explore our app and other docs in the sidebar to see all you can do with Banana!

Last updated