We believe empowering engineers drives innovation.

Aws Articles

Prior to joining Rearc, I worked as a cloud support engineer at AWS specializing in container services — specifically EKS. Of the many interesting cases I had to solve for customers during my time there, the ones that stand out are DNS resolution related issues. I am going to share the strategy I used to troubleshoot these issues and common root causes. VPC DNS Throttling issue Let’s say you have an EKS cluster with some workload running on it.

By Sam Babaiants
May 14, 2024

In August of 2023 AWS introduced Security Groups for Network Load Balancers and now recommend that any new NLBs be associated with a Security Group. They do not state the obvious: you should stop using NLBs without Security Groups. That may be easier said than done. Here is some advice with examples in Terraform and Python. AWS does not allow us to transform an NLB which has no Security Groups into one which does nor vice versa.

By Henry Brown
April 30, 2024

At Rearc Data, we build a wide variety of data pipelines from hard-to-process sources. These include PDF reports, web dashboards, and free text. Often, there is some pattern or visual guide that enables us to extract data cleanly. Other times, the data we want is entirely unstructured in that context, and there’s no “right” way to extract it. To illustrate this problem, let’s use a real-world use case. One of our Health and Life Sciences (HLS) datasets pulls weekly influenza data from a PDF report published by a Spanish agency.

By David Maxson
April 17, 2024

Large language models (LLMs) have introduced a whole new world of opportunities. Suddenly, the interface between machines and human communication has broken down, and ideas that were recently untenable have become almost trivially simple. It’s an exciting time, for sure. Real success, however, depends on wrapping this new technology with software that makes it truly useful. One pattern that has arisen to make that happen is Retrieval Augmented Generation (RAG).

By David Maxson
March 26, 2024

A picture is worth a thousand words - Fred R. Barnard In the rapidly changing world of LLMs, chatbots, and RAGs (Retrieval-Augmented Generation), most of the focus has been on plain text data. However, a wealth of information is available only in image form. What does something look like? How do elements of a diagram fit together? How is the food pyramid laid out? How do two algorithm outputs vary in visual quality?

By David Maxson
March 8, 2024

Introduction Integrating Azure Active Directory (AD), now called Microsoft Entra ID Connect, and AWS IAM Identity Center to enable single sign-on (SSO) can be a long and unclear process. One of the necessary steps in this process is configuring Azure AD to send information to AWS IAM Identity Center that it requires, when the two exchange information (referred to as provisioning in Azure AD). If this step is not performed correctly, provisioning will fail.

By Patrick Russell
September 18, 2023

What is Cloud Governance? Cloud governance includes the policies, procedures, and controls to manage cloud resources. They ensure that organizations use resources in line with their goals and objectives. Why Do You Need Cloud Governance Cloud governance has many benefits, including: Cost Optimization: Cloud computing’s scalability makes increasing or decreasing resources easy. The ease of scalability can result in wasteful expenditures if you do not actively remove unused resources. Cloud governance monitors usage and identifies savings opportunities, helping organizations optimize their budget.

By Elizabeth Simonian
July 31, 2023

Introduction & Background Hi, I’m Mueez Khan. I’m currently an undergraduate Computer Science student at Rutgers University, and I was a remote summer intern at Rearc from June to September, 2022. In this article we’ll go over various projects and achievements during my internship at Rearc. About Rearc Rearc is a boutique cloud software & services firm with engineers that have years of experience shaping the cloud journey of large scale enterprises.

By Mueez Khan
October 10, 2022

Prompted by an uptick in high profile outages, one product engineering team at a global investment bank began to explore options for increasing the resiliency of their services. With a certainty of rapid growth as the team prepares to onboard product data and consumers from several disparate teams across the firm, the team began to look towards a migration to Amazon Web Services for their unified platform. Comprised of a small handful of specialized software engineers, experts in their domains but unfamiliar with cloud infrastructure, the product team engaged with Rearc to solve their needs.

By Will Kabrich
September 16, 2022

Terraform is no doubt a popular go-to for developers looking to manage their infrastructure as code. It’s powered by HCL, a declarative configuration language from HashiCorp, that in many cases offers a powerful way to deploy infrastructure where the author can just focus on the “what” (desired state) and let the tool figure out the “how” to reconcile changes needed in the live environment. Sometimes though, operations personnel require more precise control over how changes get applied in order to minimize impact to live systems.

By Kyle Persohn
September 2, 2022

Flow logs are the native network logging layer for AWS. These logs can be setup specifically for logging IP traffic on subnets, network interfaces, or VPCs. VPC flow logs in particular contain a vast amount of IP traffic information and data points for our resources that can be leveraged for: Monitoring boundaries for networks and AWS accounts Detecting anomolous network activity Catching unintentional cross-region data transfers early (to avoid unnecessary costs) Identifying system optimizations based on AZ distribution Performing various network traffic flow optimizations In this blog post, we’ll be learning how to:

By Mueez Khan & James Becker
August 18, 2022

Introduction Migrations are one of the primary focuses for a company when they’ve decided to move to the cloud. Not only is it a shift in who is hosting your data and applications, but is also a phenomenal opportunity to modernize, optimize, and fundamentally improve the technology stacks that support business processes. This blog post details some of the benefits of moving to the cloud along with some tips to make the move as cost-effective as possible.

By Nick Kitmitto
August 3, 2022

Performing any data migration at scale is a delicate calculation of technical complexities and cost. AWS has an extensive suite of database services to assist with these efforts in their Relational Database Service (RDS) offerings. In this post, we discuss the Aurora Global Database product and how it was used to migrate applications for a large financial media customer. We discuss the pros and cons of the product as it relates to other RDS offerings and why this was the best solution for our use case.

By Lawrence Aiello
July 26, 2022