Python serverless server
How can we store and provide access to password related information?
Introduction
Given that we want to build our own password manager and deploy it in AWS (as motivated in the introduction), how should we build and deploy our server component if we want it to be free (if we're the only ones that are going to use it) or low cost (if others are going to use it as well) over the months and years to come? The general answer to that is that we want to use serverless computing, because then we don't have to pre-provision any server side resources and can thus avoid paying for computing resources that we don't actually need (note that serverless doesn't intend to imply that no servers are involved). The specific answer to that is that I chose to use the Python programming language to code the server behavior and the AWS chalice microframework to deploy it to the AWS API Gateway and Lambda services, both of which are serverless. And, at runtime, the Python server behavior interacts with the AWS Cognito, SES, and DynamoDB services, all of which are also serverless. Thus, we can deploy a server component without having to pre-provision any server resources.
Context
When I first started looking at AWS' serverless technologies, back in 2017 or so, I had a hard time seeing how teams in general and our teams in particular could take advantage of them for providing API endpoints that were tied to some backend behavior, if a lot of endpoints were necessary. The reason I had a hard time with this was that the demos and documentation that I looked through at the time showed how to create and maintain individual API endpoints and the Lambda functions that would back them. But, what if we needed to coordinate the release of tens, hundreds, or thousands of these API endpoint and Lambda function pairs?
However, the chalice microframework (and others like it) have filled that gap that was a stumbling block to me back then. It allows us to describe our server behavior in a set of Python files and then deploy all of the API gateway and Lambda function pairs at the same time. And, as a nice bonus, it allows us to describe our server behavior in a way that is quite compatible with the popular Flask web application framework for Python. That helps to allow us to test our server behavior locally, with a local Flask server, if we want to do so.
Problem
The essence of the problem that we want to solve with this server behavior is to provide a client with the relevant cloud account salt value and password transformation information, when they ask for it (as described in the introduction). This is the server interaction that will provide the most value to the user, because it will allow their client to derive the password and then allow them to copy it and then paste it in to their cloud provider's authentication component so that they can log in to that cloud account. And, this interaction is the one that will be executed the most often, with the assumption that once these cloud account representations are set up in the server, their values will be read/downloaded from the server many times for every time that they are written/uploaded.
For example, a typical user session might include getting their password for their Mint account, logging in to Mint to find out how much they owe for all of their credit cards, then getting their password for their bank, logging in to that bank account to set up the transfers/payments for those credit cards, and then doing the same sorts of things to set up transfers/payments for their utilities. Each time the user needs a password for a particular cloud account of theirs, their client will request this information from the server, derive the password with the help of this information, and then make the password available to the user.
Surrounding the essence of the problem is that we need to make sure that these API endpoints are secure, in the confidentiality, integrity, and availability (CIA) senses of the word secure. With respect to confidentiality, we need to make sure that this password related information is only exposed to the person that owns it. With respect to integrity, we want to make sure that only the person that owns this information can change it. And with respect to availability, we want to make sure that the person that owns this information is always able to access it.
Therefore, we want our solution to employ the latest and greatest tools and techniques related to security. AWS Cognito and other third party services allow us to offload at least part of the authentication behaviors we want when we expose API endpoints to the internet. They support two-factor authentication (2FA) and multi-factor authentication (MFA) and they do so with guaranteed levels of service. Thus, we should take advantage of them if we can. And, JavaScript Web Tokens (JWTs) have helped to advance the security of API endpoints by allowing tokens to be signed and validated, in addition to having randomly generated identifiers. So, we should leverage them too.
Finally, although they are not the essence of the problem that we're trying to solve either, we need to provide the user with the ability to manage the information associated with their cloud accounts in Rosetta Salt; we can think of the information in Rosetta Salt that represents a cloud account for a particular user as a target site. So, we need to allow their client applications to add new target sites, update existing target sites, and delete existing target sites. These additional behaviors are necessary to enable the steady-state use of the server, even though the frequency of these operations will likely be much lower than the actual steady-state operations (getting the information for target sites and for a particular target site).
Solution
Multi-Factor Authentication via Cognito, DynamoDB, and SES integration
Even though it surrounds the essence of the problem, I will first describe the solution for securing the endpoints. I chose to use Cognito and its email/password authentication mechanism as the first factor for authenticating with the server; this first factor results in a Cognito JavaScript Web Token (JWT) being issued to the client. I then chose to use a custom authentication mechanism as the second factor for authenticating with the server; this second factor validates the Cognito JWT that is passed to it and then uses SES to email a random code to the user which then allows them to claim a Rosetta JWT, which can then be used to interact with all of the Rosetta Salt API endpoints. The Cognito JWT can remain valid for 30 days and can be refreshed on the client during that time. But, the Rosetta JWT is only valid for 12 hours and cannot be refreshed...this forces the user to obtain a new claim code through their email account each day.
The reason that I chose to handle the second factor myself, rather than allowing Cognito to handle both factors, came down to a potential cost issue related to SMS. When MFA is enabled for Cognito, phone numbers must be verified (which could cause one-off SMS costs) and I could not find a way to force the second factor to be email during the normal flow of MFA with Cognito (which could cause regular SMS costs). In contrast, using AWS SES to send email is more predictably cheaper than potentially sending some mixture of email and SNS messages via Cognito. However, I would have chosen Cognito to handle both factors if this had not been a potential problem. And if you're aware of how to do so, I would recommend that approach above mine (and if you would, please let me know how and I'll incorporate your knowledge into this article). Therefore, to keep this security aspect of the solution free, I used Cognito for the first factor (email/password to demonstrate something the user knows) and Rosetta Salt for the second factor (receiving a code via email to demonstrate something the user has).
Below are some relevant snippets of my chalice app.py file, along with some links to pages with more detail on its various parts. Also, an accompanying YouTube video that shows how to deploy the chalice application can be seen here.
The first factor of the authentication process cannot be seen here. That is because it is completed with the Amplify JavaScript framework for React or React Native on the different clients and they interact directly with Cognito. This part of the solution is described in the client sections of the article. After the first stage of authentication is complete, the client will have a Cognito JWT and will present it to the server while it is creating a pending Rosetta token, and then while it is claiming that token.
The second factor of the authentication process can be seen here. The chalice server app validates this Cognito token before it creates a pending Rosetta token in DynamoDB and then sends out an email to the user via SES. And, it validates this Cognito token before it claims this pending Rosetta token for the user and then creates an operational token for them in DynamoDB. Finally, a bearer token (the Rosetta JWT) representing the operational token is returned to the client for the user to use throughout the rest of their work day.
--
app.py
--
from chalice import Chalice
from chalicelib import requestUtility
app = Chalice(app_name='rosetta-api-chalice')
#
# Convenience functions
#
def validateCognitoToken(requestHeaders):
return requestUtility.RequestUtility().validateCognitoToken(app, requestHeaders)
#
# API endpoints - token-requests
#
@app.route('/token-requests', methods=['POST'])
def createPendingToken():
userId = validateCognitoToken(app.current_request.headers)
tokenRequestInfo = app.current_request.json_body
pendingTokenId = requestUtility.RequestUtility().createPendingToken(userId, tokenRequestInfo)
return pendingTokenId
@app.route('/token-requests/{tokenId}', methods=['PATCH'])
def claimPendingToken(tokenId):
userId = validateCognitoToken(app.current_request.headers)
tokenClaimInfo = app.current_request.json_body
bearerToken = requestUtility.RequestUtility().claimPendingToken(tokenId, userId, tokenClaimInfo)
return bearerToken
--
Cloud account salt value and password transformation information via DynamoDB
Now that you understand the authentication process (described above), you can see that the user (via their client) will need to present a Rosetta JWT when they want to use any of the other API endpoints. And, again, the essence of the problem that we want to solve with this server behavior is to provide a client with the relevant cloud account salt value and password transformation information, when they ask for it (and only when they ask for it), so that they can derive their password for that cloud account and log in to it.
Below are some relevant snippets of my chalice app.py file, along with some links to pages with more detail on its various parts.
Before a user can get the specific information they need for a particular cloud account (or target site) they need to retrieve a list of the cloud accounts (or target sites) that they have previously configured within Rosetta Salt. When they request that list, via their client, the chalice server app validates their Rosetta token and then gets all of the target site information for that user from DynamoDB, and then returns a list of them that the user can look at with their client.
Then, once the user can see the list of their target sites, they will most often select one of them and request the salt value and the password transformation information for it. When they request that specific information, via their client, the chalice server app validates their Rosetta token and then gets that target site information from DynamoDB, and then returns it so that the user can look at it with their client, derive their password for that cloud account, and then log in to that cloud account.
--
app.py
--
from chalice import Chalice
from chalice import UnauthorizedError
from chalicelib import requestUtility
app = Chalice(app_name='rosetta-api-chalice')
#
# Convenience functions
#
def validateUser(requestHeaders):
tokenItem = requestUtility.RequestUtility().validateBearerToken(app, requestHeaders)
if tokenItem is None:
raise UnauthorizedError('Expired or invalid token identified')
userId = tokenItem.get('userId')
if userId is None or len(userId) <= 0:
raise UnauthorizedError('Expired or invalid token identified')
return userId
#
# API endpoints - target-sites
#
@app.route('/target-sites/{siteId}')
def getTargetSite(siteId):
userId = validateUser(app.current_request.headers)
return requestUtility.RequestUtility().getTargetSite(userId, siteId)
@app.route('/target-sites')
def getTargetSites():
userId = validateUser(app.current_request.headers)
return requestUtility.RequestUtility().getTargetSites(userId)
--
Maintain cloud account information via DynamoDB
Once the server endpoints are secure and the primary user workflow is supported, as described above, the more mundane but nonetheless necessary secondary workflows need to be supported. That is, the user needs to be able to add new target sites, delete existing target sites, and sometimes they need to be able to update existing target sites.
Below are some relevant snippets of my chalice app.py file, along with some links to pages with more detail on its various parts.
Before a user can add, update, or delete a target site, they need to have gone through the authentication process so that we only allow a user to maintain their own cloud accounts information. When they request to add, update, or delete a target site, the chalice server app validates their Rosetta token and then adds, updates, or deletes a target site by interacting with DynamoDB, and then returns any appropriate information to them via their client.
--
app.py
--
from chalice import Chalice
from chalicelib import requestUtility
app = Chalice(app_name='rosetta-api-chalice')
#
# Convenience functions
#
def validateUser(requestHeaders):
tokenItem = requestUtility.RequestUtility().validateBearerToken(app, requestHeaders)
if tokenItem is None:
raise UnauthorizedError('Expired or invalid token identified')
userId = tokenItem.get('userId')
if userId is None or len(userId) <= 0:
raise UnauthorizedError('Expired or invalid token identified')
return userId
#
# API endpoints - target-sites
#
@app.route('/target-sites', methods=['POST'])
def createTargetSite():
userId = validateUser(app.current_request.headers)
inputSiteInfo = app.current_request.json_body
newSiteMemento = requestUtility.RequestUtility().createTargetSite(userId, inputSiteInfo)
return newSiteMemento
@app.route('/target-sites/{siteId}', methods=['DELETE'])
def deleteTargetSite(siteId):
userId = validateUser(app.current_request.headers)
return requestUtility.RequestUtility().deleteTargetSite(userId, siteId)
@app.route('/target-sites/{siteId}', methods=['PATCH'])
def updateTargetSite(siteId):
userId = validateUser(app.current_request.headers)
inputSiteInfo = app.current_request.json_body
return requestUtility.RequestUtility().updateTargetSite(userId, siteId, inputSiteInfo)
--