An Interest In:
Web News this Week
- April 19, 2024
- April 18, 2024
- April 17, 2024
- April 16, 2024
- April 15, 2024
- April 14, 2024
- April 13, 2024
April 19, 2022 02:09 am GMT
Original Link: https://dev.to/wesleycheek/saveload-tensorflow-sklearn-pipelines-from-local-and-aws-s3-34dc
Save/Load Tensorflow & sklearn pipelines from local and AWS S3
After a lot of struggle doing this, I finally found a simple way.
We can write and read Tensorflow
and sklearn
models/pipelines using joblib
.
Local Write / Read
from pathlib import Pathpath = Path(<local path>)# WRITEwith path.open("wb") as f: joblib.dump(model, f)# READwith path.open("rb") as f: f.seek(0) model = joblib.load(f)
We can do the same thing on AWS S3 using a boto3
client:
AWS S3 Write / Read
import tempfileimport boto3import joblibs3_client = boto3.client('s3')bucket_name = "my-bucket"key = "model.pkl"# WRITEwith tempfile.TemporaryFile() as fp: joblib.dump(model, fp) fp.seek(0) s3_client.put_object(Body=fp.read(), Bucket=bucket_name, Key=key)# READwith tempfile.TemporaryFile() as fp: s3_client.download_fileobj(Fileobj=fp, Bucket=bucket_name, Key=key) fp.seek(0) model = joblib.load(fp)# DELETEs3_client.delete_object(Bucket=bucket_name, Key=key)
Original Link: https://dev.to/wesleycheek/saveload-tensorflow-sklearn-pipelines-from-local-and-aws-s3-34dc
Share this article:
Tweet
View Full Article
Dev To
An online community for sharing and discovering great ideas, having debates, and making friendsMore About this Source Visit Dev To