AWS Application Load Balancers and Ignition

What is a Load Balancer?

A Load Balancer (LB) provides an efficient way to distribute traffic and increase performance between servers and clients. When large enterprise systems are expected to receive large amounts of clients daily, they may reach a point where a single server will not be enough to handle all requests. With a LB, multiple servers can be utilized to handle the same client requests at a much larger scale. Depending on the current client load of each server, the load balancer will decide which server is best to receive additional incoming client connections.

What are the main types of Load Balancers?

The two main types of LBs will distribute traffic at either the Network or Application layer of communication. 

Network LBs operate at the fourth layer of the OSI model or the Transport layer, meaning we focus on TCP/UDP connections that are established between specific source and destination IPs and Ports. So in terms of client requests making their way to servers, the LB only decides what destination IP is best for a connection without any additional context.

Application LBs operate at the seventh layer of the OSI model or the Application layer, meaning we focus on the interaction with a specific software application and its client requests via HTTP/HTTPS. When a client requests from a server, the LB will instead decide based on multiple-defined http/https endpoints to establish a connection.

AWS Application Load Balancers

Cloud-based, scaled-out architectures have become a very popular option for deploying larger enterprise systems. When multiple Perspective sessions and/or Vision clients need to be able to reach the Ignition server through this setup, you may need to ensure all of those connections are established and stay healthy without potentially interfering with one another. An AWS Application Load Balancer (using the application layer) is an option provided by the Elastic Load Balancing feature and will allow you to efficiently distribute traffic of individual sessions/clients across your cloud instances.

AWS Configuration

Assuming you already have properly configured EC2 instances to work with that already contain Ignition servers using SSL, we can navigate through your EC2 Dashboard and navigate to Load Balancing/Load Balancers to begin your configuration. (Note: It is important that the Ignition servers themselves have SSL configured. Any attempts to have the LBs point to a non-https route / unsecure route will fail).

Step 1:

1. Specify a Name for this LB
1. The Name specified here will define the DNS hostname used to establish all sessions with our clients. So make sure you specify a meaningful name that you want your clients to see. 
2. Specify the Scheme as Internet-facing
3. IP address type: IPv4 
4. Under Listeners, add a listener with
1. Load Balancer Protocol: HTTPS
2. Load Balancer Port: 8043
5. Under Availability Zones
1. Leave VPC as default
2. Specify the Availability Zones that each of your EC2 instances reside in. If you’re not too sure about this, check the Instances page any look at the Availability Zones that are set for each of them
1. This will allow the LB to look in the right place for each of the instances, so make sure none are left out here.

Step 2:

For this section, we will need an SSL certificate applied to the connection between our clients and the LB itself. For this example I will be importing one using IAM based on my permissions and preferences. In your case you may have a different set of permissions on your AWS account and may be instructed to use ACM for storing and retrieving your certificates.

If someone has already created these certificates for you, then an IAM role with the correct permissions should be able to see previously uploaded certificates after selecting Choose a certificate from IAM. If you need to import one for the first time, you will need to create one from scratch Upload a certificate to ACM or IAM.

Step 3:

For this section, you also may already have some existing Security Groups defined when first setting up your instances. If not, you will need to confirm what ports are being used for your external connections and add them to the list of Inbound Rules when creating a new group.

For this setup example, ensure that you at least have port 8043 added

1. Rule: Custom TCP Rule
2. Protocol: TCP
3. Port Range: 8043
4. Source: 0.0.0.0/0

Step 4:

Under target groups, you are defining how the load balancer will attempt to reach your existing EC2 instances.

1. Target group: New target group
2. Name: {Your Preferred Name}
3. Target type: Instance
4. Protocol: HTTPS
5. Port: 8043
6. Health Checks
1. Protocol: HTTPS
2. Path: /system/gwinfo
7. Advanced health check settings
1. Success codes: 200,302

Step 5:

Here we are selecting EC2 instances that will be involved with the LB. When selected, the instances will be registered to the Target Group that you created in the previous step. You can always add the instances later after the LB is created by navigating to Target Groups/{Your Target Group}/Register Targets.

Step 6:

Review the settings and hit the Create button once you are done. The final step under the EC2 configuration will be to head to Target Groups/{Your Target Group}/Attributes.

Edit Attributes, enable Stickiness and set your preferred duration. This setting will decide how long a vision client / perspective session will stay under a specific EC2 instance that exists under your Target Group’s Registered Targets.

After this step, the load balancer should be ready to receive client / session requests from Ignition. Download a designer, vision client launcher, workstation launcher, or open browser to enter the URL: https://{YourLoadBalancerDNSName}:8043. You can find the DNS Hostname at the Load Balancers page under the Descriptions tab. 

If everything is set up properly, you should be able to establish a connection using that URL and will be redirected to one of it’s registered targets. For all launchers, expect a prompt to trust certificates and approve it.

Troubleshooting Guidelines

• The EC2’s Load Balancer page contains a Monitoring tab to look for any undesirable HTTP response codes, Issues connecting to Registered Targets, and Response times of the Load Balancer itself.

• You cannot use Wireshark between the load balancer and registered targets, but you can at least monitor traffic from a client to the LB itself. 
• Capture filter: ip host X.X.X.X and {YourDNSHostname}
• You need to depend on the Monitoring tab from the Load Balancer page to look for internal EC2 connection issues.
• Ensure all individual instances contain the same project configuration
• If, for any reason, a project is missing/misconfigured on one of the Registered Targets and not others, then you run the risk of failed/unmatched clients / sessions if the LB decides to select that instance.
• If you are receiving any permission errors when uploading SSL certificates, creating Target Groups, etc. to an AWS LB configuration, this will require changes made by the administrator for your AWS account and it’s roles.
• These permissions can also affect visibility of previously created certificates and profiles.
• Look out for errors such as User: ___ is not authorized to perform: ___ on resource: ___ when making configuration changes.

Additional Notes

• When a Designer, Workstation, Vision instance is created in their launchers, you may see a unique Gateway name on the launchable instance if all servers are not given the same name
• When this happens, do not expect the launchable instance to consistently navigate to that particular server
• Each time a launchable instance is created and selected to launch, it is always up to the LB to decide where the destination will be since it is still a new connection
• For example, tag server 3 launchable instance still can potentially direct you to tag server 1 or 2 each time a launch is made
• In Perspective, refreshing the browser, opening new tabs, new windows creates new pages but doesn’t make new sessions. Only a new incognito/private tab creates a completely new session on the same machine. This is expected Ignition behavior.
• For the load balancer, since perspective retains the IP/Machine information for a session, no matter how many pages are opened from one machine, the load balancer will treat all pages as one session and will not need to check other back end servers each time a new page is opened to the same project.
• This does not apply to incognito mode though. Since using incognito mode requests a new perspective session, the load balancer will treat this as a new request and potentially direct us to a different backend server.