I have a FastAPI backend that needs to load new deep learning models everyday to memory. Models loads for the first time when FastAPI starts or force reload.
I have a class Predict and an endpoint to get data models info (mock code):
controller.py
def model_init():
# vectors = <some code>
# labels = <some code>
return vectors, labels
def update(date):
# <some code to get new models from server by date>
return vectors, labels
class Predict:
def __init__(self, vectors, labels) -> None:
self.vectors = vectors
self.labels = labels
self.vectors, self.labels = model_init()
def updateVectors(self, new_vector):
self.vectors = new_vector
def updateLabels(self, new_label):
self.labels = new_label
def process(self, data):
print("Using models with vectors length = ", len(vectors), ", labels length = ", len(labels))
# some code using self.vectors, self.labels
# response = <something>
return response
view.py
from controller import model_init
from datetime import datetime
vectors, labels = model_init()
predict = Predict(vectors, labels)
@router.get("/info")
async def get_info():
response = {
"vectors": len(vectors),
"labels": len(labels),
"date": datetime.now().strftime("%Y%m%d")
}
return {"info": response}
@router.post("/predict")
async def predictor(data):
try:
response = predict.process(data)
return {"result": response}
except PredictExceptionError as e:
raise e
Suppose that, today I start Uvicorn FastAPI backend (uvicorn main:app --reload), after waiting for models first loaded to memory, I run /info endpoint. Result is len(vectors) = 1000000, and len(labels) = 1000.
What I am expecting is, when running /info endpoint again tomorrow, it returns len(vectors) = 2000000, and len(labels) = 2000. Also, when running /predict endpoint, new models have to load to memory without reloading FastAPI.
Any methods or approaches to update models to memory?
predictreference).