Customizing
In this page, we'll customize our Potassium app.
Iteration speed developing AI/ML backends can feel very slow at times, due to reloading the model into memory on every code change.
Thankfully, the Banana CLI dev server includes Hot Reload, which watches your
app.py for changes and reloads the models if the init block changes
See it for yourself by changing the
@app.handler() logic to return all outputs from the inference, rather than just the 0th indexfrom:
return Response(
json = {"outputs": outputs[0]},
status=200
)
to:
return Response(
json = {"outputs": outputs},
status=200
)
Save
app.py and you'll see the dev server update immediately------
Hot reloading 🔥
Reloaded
------
Then calling the model:
curl -X POST -H "Content-Type: application/json" -d '{"prompt": "Hello I am a [MASK] model."}' http://localhost:8000/
you'll see the new handler logic run
{"outputs":[
{
"score": 0.13177461922168732,
"sequence": "hello i am a fashion model.",
"token": 4827,
"token_str": "fashion"
},
{
"score": 0.1120428815484047,
"sequence": "hello i am a role model.",
"token": 2535,
"token_str": "role"
},
...
{
"score": 0.022045975551009178,
"sequence": "hello i am a model model.",
"token": 2944,
"token_str": "model"}
]}
In the next page, we'll use the clientside SDKs to call our Potassium server
Last modified 1mo ago