Automation

Data Acquisition

The web server will be deployed on the Google Cloud Run and be scheduled to fetch relevant data from Iron Hand repeatedly, every hour. It will upload the apy data in a readable format.

Data Ingestion and Preprocessing

In order to start the process of model training, the readable file in GCS will be loaded and preprocessed in order to prepare high quality data for training. The preprocess phase will clean the data, commit feature engineering in order to generate new features to enhance the model’s predictive power, normalizing the data, splitting the data into training, validation and test sets etc.

Hyperparameter Tuning

Tuning hyperparameters like number of neurons, activation function, learning rate, batch size, loss function, epochs etc to boost the performance of the model.

Model Training

In this phase, define the LSTM model, train it, make predictions and evaluate the model.

Model Evaluation

Use the test set to evaluate the performance of the model and make sure the model’s performance surpasses the certain threshold, save it into GCS.

Model Deployment

Deploy the saved model for serving prediction data.

Continuous Monitoring

Monitor the health and performance of the deployed model and set up alerts for any anomalies or drift in the model performance.

Feedback & Retraining

If the monitoring phase detects drift or decreased performance, retrain the model.