How to Train and Deploy a Sentiment Analysis Web App
In this article, You will learn how to deploy a sentiment analysis Model and access it on a webpage using AWS
We will be taking a look at how we can use a model created in SageMaker to create an endpoint that will be used as a way to send data to our model. An endpoint is basically a way to allow a model and an application to communicate. An application, such as a web app will be responsible for accepting user input data. Through an endpoint, we can send that data to our model, which will produce predictions that can be sent back to our application. We are going to be looking at a web app that will interact with a deployed model.
Note: If you do not have an AWS account, please follow the instructions in this article on how to set up one.
In this step-by-step tutorial, i will be walking you through how to use Amazon SageMaker to create an LSTM model to try and determine whether a review posted on the IMDB Database is positive or negative using a web app that returns the sentiment of a movie review. This model is trained on the IMDB movie review dataset.
General Outline
- Download or otherwise retrieve the data.
- Process / Prepare the data.
- Upload the processed data to S3.
- Train a chosen model.
- Test the trained model (typically using a batch transform job).
- Deploy the trained model.
- Use the deployed model.
Note: This is a Lengthy step-by-step explanation of my solution to one of my Machine Learning Udacity projects which were deploying a sentiment analysis web app on. The link to this GitHub repo can be found here.
First, we create a directory, download, and save the IMDB Dataset used for binary sentiment classification.
Now that we’ve read the raw training and testing data from the downloaded dataset, we will combine the positive and negative reviews and shuffle the resulting records. We still have to go through the data and check the data to see what our model will be trained on and show examples if the data has been loaded correctly. This is a good practice as it allows you to see how each of the further processing steps affects the reviews. After we are fine that the data has been prepared properly. We move on to Processing.
The first step in processing the reviews is to ensure that any Html tags that appear should be removed. The review_to_words method below uses BeautifulSoup to remove any Html tags that appear and uses the nltk package to tokenize the reviews. It does this by reducing every word in the review to its root word. e.g., words like “chocolates,” “chocolatey” -> “choco” to the root word.
To ensure we know how everything is working, We apply review_to_words to one of the reviews in the training set. Note: This processing step usually takes a long time, so be aware of that
The method below applies the review_to_words method to each of the reviews in the training and testing datasets. In addition, it caches the results. This is because performing this processing step can take a long time. This way, if you are unable to complete the notebook in the current session, you can come back without needing to process the data a second time.
Next, we are going to be transforming the data.
Transform the data
Here we are going to transform the data from its word representation to a bag-of-words feature representation. In the model we will construct in this notebook, we will construct a feature representation by representing each word as an integer. Of course, some of the words that appear in the reviews occur infrequently and so likely don’t contain much information for sentiment analysis. The way we will deal with this problem is that we will fix the size of our working vocabulary, and we will only include the words that appear most frequently. We will then combine all of the infrequent words into a single category and, in our case, we will label it as 1. Since we will be using a recurrent neural network, it will be convenient if the length of each review is the same. To do this, we will fix a size for our reviews and then pad short reviews with the category ‘no word’ (which we will label 0) and truncate long reviews.
To begin with, we need to construct a way to map words that appear in the reviews to integers. Here we fix the size of our vocabulary (including the ‘no word’ and ‘infrequent’ categories) to be 5000 But you may wish to change this to see how it affects the model. We complete the implementation for the build_dict() method below.
build_dict() MethodNote that even though the vocab_size is set to 5000, we only want to construct a mapping for the most frequently appearing 4998 words. This is because we want to reserve the special labels 0 for 'no word' and 1 for 'infrequent word.'
Transform the reviews
Now that we have our word dictionary, which allows us to transform the words appearing in the reviews into integers, it is time to make use of it and convert our reviews to their integer sequence representation, making sure to pad or truncate to a fixed length, which in our case is 500.
Step 3: Upload the data to S3
When a training job is constructed using SageMaker, a container is executed which performs the training operation. This container is given access to data that is stored in S3. This means that we need to upload the data we want to use for training to S3 using the session object associated with this notebook. We will need to upload the training dataset to S3 for our training code to access it. Firstly we save the data locally then upload it to S3. Suppose you want to find out more about S3 in general. Please read my article on Amazon SageMaker in a nutshell.
NOTE: The cell above uploads the entire contents of our data directory. This includes the word_dict.pkl file. This is fortunate as we will need this later when creating an endpoint that accepts an arbitrary review. For now, we will just take note of the fact that it resides in the data directory (and so also in the S3 training bucket) and that we will need to make sure it gets saved in the model directory.
Step 4: Build and Train the PyTorch Model
In the SageMaker framework. In particular, a model comprises three objects
- Model Artifacts,
- Training Code, and
- Inference Code,
We will start by implementing our own neural network in PyTorch along with a training script. For this project, we have provided the necessary model object in the model.py file inside of the train folder. You can see the provided implementation by running the cell below.
We constructed the LSTM class that inherits from the nn.Module. We used the nn.Embedding module to create this layer, which takes the vocabulary size and desired word-vector length as input. We then provided a padding index to indicate the index of the padding element in the embedding matrix. We then passed the embedding layer’s output into an LSTM layer (created using nn.LSTM), which takes as input the word-vector length and length of the hidden state vector. Then we pass it through the linear layer and finally through the sigmoid activation function to get the probability of the sequences before we perform a forward pass of our model on the input.
The important takeaway from the implementation provided is that there are three parameters that we may wish to tweak to improve the performance of our model. These are the embedding dimension, the hidden dimension, and the size of the vocabulary. We will likely want to make these parameters configurable in the training script so that if we wish to modify them, we do not need to modify the script itself. We will see how to do this later on. To start, we will write some of the training code in the notebook to more easily diagnose any issues.
First, we will load a small portion of the training data set to use as a sample. It would be very time-consuming to try and train the model completely in the notebook as we do not have access to a GPU, and the compute instance that we are using is not particularly powerful. However, we can work on a small bit of the data to understand how our training script is behaving.
Here we downloaded the first 250 rows of the data and used pandas Dataframe to turn the NumPy arrays into tensors, building the sample dataset and passing it to Pytorch Data loader to specify a batch size. Writing the training method.
Next, we need to write the training code itself.
Now that we have the training method above, we will test it by writing a bit of code in the notebook that executes our training method on the small sample training set that we loaded earlier. The reason for doing this in the notebook is to have an opportunity to fix any errors that arise early when they are easier to diagnose.
We can see, based on the sample data provided, the BCELoss, or Binary Cross-Entropy Loss reducing, so that means we are on the right path. Note: Binary Cross-Entropy Loss is a kind of cross-entropy loss designed to work with a single Sigmoid output(a single value between 0 and 1).
Training the model
To construct a PyTorch model using SageMaker, we must provide SageMaker with a training script. We may decide to include a directory that will be copied to the container and from which our training code will be run. When the training container is executed, it will check the uploaded directory (if there is one) for a requirements.txt file and install any required Python libraries, after which the training script will be run.
When a PyTorch model is constructed in SageMaker, an entry point must be specified. This is the Python file that will be executed when the model is trained. Inside of the train directory is a file called train.py which has been provided and which contains most of the necessary code to train our model. The only thing that is missing is the implementation of the train() method which you wrote earlier in this notebook.
The way that SageMaker passes hyperparameters to the training script is by way of arguments. These arguments can then be parsed and used in the training script. To see how this is done, take a look at the provided train/train.py file.
Step 5: Testing the model
As mentioned at the top of this notebook, we will be testing this model by first deploying it and then sending the testing data to the deployed endpoint. We will do this so that we can make sure that the deployed model is working correctly.
Step 6: Deploy the model for testing
Now that we have trained our model, we would like to test it to see how it performs. Currently, our model takes the input of the form review_length, review[500] where review[500] is a sequence of 500 integers which describe the words present in the review, encoded using word_dict. Fortunately for us, SageMaker provides built-in inference code for models with simple inputs such as this.
There is one thing that we need to provide, however, and that is a function that loads the saved model. This function must be called model_fn() and takes as its only parameter a path to the directory where the model artifacts are stored. This function must also be present in the python file, which we specified as the entry point. In our case, the model loading function has been provided, so no changes need to be made.
NOTE: When the built-in inference code is run, it must import the model_fn() method from the train.py file. This is why the training code is wrapped in the main guard (i.e., if __name__ == '__main__': )
Since we don’t need to change anything in the code uploaded during training, we can simply deploy the current model as-is.
NOTE: When deploying a model, you ask SageMaker to launch a compute instance that will wait for data to be sent. As a result, this compute instance will continue to run until you shut it down. This is important to know since the cost of a deployed endpoint depends on how long it runs.
In other words, If you are no longer using a deployed endpoint, shut it down!
Once deployed, we can read the test data and send it off to our deployed model to get some results. Once we collect all of the results, we can determine how accurate our model is.
Note: It's important to note that this model’s hyperparameter has not been tuned yet to get the best performance yet, so there is still room for the accuracy score to improve. We now have a trained model deployed and which we can send processed reviews and which returns the predicted sentiment. However, ultimately we would like to be able to send our model an unprocessed review. That is, we would like to send the review itself as a string. For example, suppose we wish to send the following review to our model.
Recall earlier in the article. We did a bunch of data processing to the IMDb dataset. In particular, we did two specific things to the provided reviews. Firstly we removed any HTML tags and stemmed the input, and then encoded the review as a sequence of integers using. word_dict .
To process the review, we will need to repeat these two steps. Therefore we will be using the review_to_words and convert_and_pad methods from section one and converting test_review into a NumPy array test_data suitable to send to our model. Remember that our model expects an input of the form. review_length, review[500].
Delete the endpoint
Now that we have tested and deployed our model and sees that it works, we need to shut down and delete the endpoint so we can work on the rest of the deployment process. The cost of a deployed endpoint is based on the length of time that it is running. This means that if you aren’t using an endpoint, you really need to shut it down.
Step 6 (again) — Deploy the model for the web app.
Now that we know that our model is working, it’s time to create some custom inference code to send the model a review that has not been processed and have it determine the sentiment of the review.
As we saw above, by default, the estimator which we created, when deployed, will use the entry script and directory which we provided when creating the model. However, since we now wish to accept a string as input and our model expects a processed review, we need to write some custom inference code.
We will store the code that we write in the serve directory. Provided in this directory is the model.py file that we used to construct our model, a utils.py file which contains the review_to_words and convert_and_pad pre-processing functions, which we used during the initial data processing, and predict.pythe file, which will contain our custom inference code.
Note also that requirements.txt is present, which will tell SageMaker what Python libraries are required by our custom inference code.
When deploying a PyTorch model in SageMaker, you are expected to provide four functions that the SageMaker inference container will use.
model_fn: This function is the same function that we used in the training script, and it tells SageMaker how to load our model.input_fn: This function receives the raw serialized input sent to the model's endpoint, and its job is to de-serialize and make the input available for the inference code.output_fn: This function takes the output of the inference code, and its job is to serialize this output and return it to the caller of the model's endpoint.predict_fn: The heart of the inference script is where the actual prediction is done and is the function you will need to complete.
For the simple website that we are constructing during this project the input_fn and output_fn methods are relatively straightforward. We only require accepting a string as input, and we expect to return a single value as output. You might imagine, though, that in a more complex application, the input or output may be image data or some other binary data, which would require some effort to serialize.
Deploying the model
Now that the custom inference code has been written, we will create and deploy our model. To begin with, we need to construct a new PyTorch Model object which points to the model artifacts created during training and also points to the inference code that we wish to use. Then we can call the deploy method to launch the deployment container.
NOTE: The default behavior for a deployed PyTorch model assumes that any input passed to the predictor is an numpy array. In our case, we want to send a string, so we need to construct a simple wrapper around the RealTimePredictor class to accommodate simple strings. In a more complicated situation, you may want to provide a serialization object, for example, if you want to sent image data.
Testing the model
Now that we have deployed our model with the custom inference code, we should test to see if everything is working. Here we test our model by loading the first 250 positive and negative reviews and send them to the endpoint, then collect the results. The reason for only sending some of the data is that the amount of time it takes for our model to process the input and then perform inference is quite long and so testing the entire data set would be prohibitive.
Now that we know our endpoint is working as expected, we can set up the web page to interact with it. If you don’t have time to finish the project now, make sure to skip down to the end of this notebook and shut down your endpoint. You can deploy it again when you come back.
Step 7 (again): Use the model for the web app
So far, we have been accessing our model endpoint by constructing a predictor object which uses the endpoint and then just using the predictor object to perform inference. What if we wanted to create a web app that accessed our model? The way things are set up currently makes that not possible since, to access a SageMaker endpoint, the app would first have to authenticate with AWS using an IAM role which included access to SageMaker endpoints. However, there is an easier way! We just need to use some additional AWS services.
The diagram above gives an overview of how the various services will work together. On the far right is the model which we trained above and which is deployed using SageMaker. On the far left is our web app that collects a user’s movie review, sends it off, and expects a positive or negative sentiment in return.
In the middle is where some of the magic happens. We will construct a Lambda function, which you can think of as a straightforward Python function that can be executed whenever a specified event occurs. We will give this function permission to send and receive data from a SageMaker endpoint.
Lastly, the method we will use to execute the Lambda function is a new endpoint that we will create using API Gateway. This endpoint will be a URL that listens for data to be sent to it. Once it gets some data, it will pass that data on to the Lambda function and then return whatever the Lambda function returns. Essentially it will act as an interface that lets our web app communicate with the Lambda function.
Setting up a Lambda function
The first thing we are going to do is set up a Lambda function. This Lambda function will be executed whenever our public API has data sent to it. When it is executed, it will receive the data, perform any sort of processing required, send the data (the review) to the SageMaker endpoint we’ve created, and then return the result.
Part A: Create an IAM Role for the Lambda function
Since we want the Lambda function to call a SageMaker endpoint, we need to ensure that it has permission to do so. To do this, we will construct a role that we can later give the Lambda function.
Using the AWS Console, navigate to the IAM page and click on Roles. Then, click on Create role. Ensure that the AWS service is the type of trusted entity selected and choose Lambda as the service that will use this role, then click Next: Permissions.
In the search box, type sagemaker and select the check box next to the AmazonSageMakerFullAccess policy. Then, click on Next: Review.
Lastly, give this role a name. Make sure you use a name that you will remember later on, for example LambdaSageMakerRole. Then, click on Create role.
Part B: Create a Lambda function
Now it is time actually to create the Lambda function.
Using the AWS Console, navigate to the AWS Lambda Page and click on Create a function. When you get to the next page, make sure that Author from scratch is selected. Now, name your Lambda function, using a name that you will remember later on, for example sentiment_analysis_func. Make sure that the Python 3.6 runtime is selected, and then choose the role that you created in the previous part. Then, click on Create Function.
On the next page, you will see some information about the Lambda function you’ve just created. If you scroll down, you should see an editor in which you can write the code executed when your Lambda function is triggered. In our example, we will use the code below.
import boto3
def lambda_handler(event, context):
# The SageMaker runtime is what allows us to invoke the endpoint that we've created.
runtime = boto3.Session().client('sagemaker-runtime')
# Now we use the SageMaker runtime to invoke our endpoint, sending the review we were given
response = runtime.invoke_endpoint(EndpointName = '**ENDPOINT NAME HERE**', # The name of the endpoint we created
ContentType = 'text/plain', # The data format that is expected
Body = event['body']) # The actual review
# The response is an HTTP response whose body contains the result of our inference
result = response['Body'].read().decode('utf-8')
return {
'statusCode' : 200,
'headers' : { 'Content-Type' : 'text/plain', 'Access-Control-Allow-Origin' : '*' },
'body' : result
}Once you copy and paste the code above into the Lambda code editor, replace the **ENDPOINT NAME HERE** portion with the name of the endpoint that we deployed earlier. You can determine the name of the endpoint using the code cell below.
Once you have added the endpoint name to the Lambda function, click on Save. Your Lambda function is now up and running. Next, we need to create a way for our web app to execute the Lambda function.
Setting up The API Gateway
Now that our Lambda function is set up, it is time to create a new API using API Gateway to trigger the Lambda function we have just created.
Using AWS Console, navigate to Amazon API Gateway and then click on Get started.
On the next page, make sure that New API is selected and give the new API a name, for example, sentiment_analysis_api. Then, click on Create API.
Now we have created an API. However, it doesn’t currently do anything. What we want it to do is to trigger the Lambda function that we created earlier.
Select the Actions dropdown menu and click Create Method. A new blank method will be created, select its dropdown menu and select POST, then click on the checkmark beside it.
For the integration point, make sure that Lambda Function is selected and click on the Use Lambda Proxy integration. This option makes sure that the data that is sent to the API is then sent directly to the Lambda function with no processing. It also means that the return value must be a proper response object as it will also not be processed by API Gateway.
Type the name of the Lambda function you created earlier into the Lambda Function text entry box and then click on Save. Click on OK in the pop-up box that then appears, permitting API Gateway to invoke the Lambda function you created.
The last step in creating the API Gateway is to select the Actions dropdown and click on Deploy API. You will need to create a new Deployment stage and name it anything you like, for example. prod.
You have now successfully set up a public API to access your SageMaker model. Make sure to copy or write down the URL provided to invoke your newly created public API, as this will be needed in the next step. This URL can be found at the top of the page, highlighted in blue next to the text Invoke URL.
Finally, We are deploying our web app.
Now that we have a publicly available API, we can start using it in a web app. For our purposes, we have provided a simple static HTML file that can use the public API you created earlier.
In the website folder, there should be a file called index.html. Download the file to your computer and open that file up in a text editor of your choice. There should be a line that contains **REPLACE WITH PUBLIC API URL**. Replace this string with the URL you wrote down in the last step and then save the file.
Now, if you open index.html on your local computer, your browser will behave as a local web server, and you can use the provided site to interact with your SageMaker model.
The above images are what to expect if the index.html file loads properly, and you can test to see if the model gives you a good review.
If you’d like to go further, you can host this HTML file anywhere you’d like, for example, using GitHub or hosting a static site on Amazon’s S3. Once you have done this, you can test the link using any review you want
Important Note In order for the web app to communicate with the SageMaker endpoint, the endpoint has to actually be deployed and running. This means that you are paying for it. Make sure that the endpoint is running when you want to use the web app but that you shut it down when you don’t need it, otherwise you will end up with a surprisingly large AWS bill.
Now that your web app is working, trying playing around with it and see how well it works.
Delete the endpoint
Remember always to shut down your endpoint if you are no longer using it. You are charged for the length of time that the endpoint is running, so if you forget and leave it on, you could end up with an unexpectedly large bill.
