Your Web News in One Place

Help Webnuz

Referal links:

Sign up for GreenGeeks web hosting
April 19, 2022 02:09 am GMT

Save/Load Tensorflow & sklearn pipelines from local and AWS S3

After a lot of struggle doing this, I finally found a simple way.

We can write and read Tensorflow and sklearn models/pipelines using joblib.

Local Write / Read

from pathlib import Pathpath = Path(<local path>)# WRITEwith path.open("wb") as f:    joblib.dump(model, f)# READwith path.open("rb") as f:    f.seek(0)    model = joblib.load(f)

We can do the same thing on AWS S3 using a boto3 client:

AWS S3 Write / Read

import tempfileimport boto3import joblibs3_client = boto3.client('s3')bucket_name = "my-bucket"key = "model.pkl"# WRITEwith tempfile.TemporaryFile() as fp:    joblib.dump(model, fp)    fp.seek(0)    s3_client.put_object(Body=fp.read(), Bucket=bucket_name, Key=key)# READwith tempfile.TemporaryFile() as fp:    s3_client.download_fileobj(Fileobj=fp, Bucket=bucket_name, Key=key)    fp.seek(0)    model = joblib.load(fp)# DELETEs3_client.delete_object(Bucket=bucket_name, Key=key)

Original Link: https://dev.to/wesleycheek/saveload-tensorflow-sklearn-pipelines-from-local-and-aws-s3-34dc

Share this article:    Share on Facebook
View Full Article

Dev To

An online community for sharing and discovering great ideas, having debates, and making friends

More About this Source Visit Dev To