Insights

Adriel Higor

### Overview
When working on data-driven projects you often need to extract data from third-party sources, public APIs, or CSV files shared by other teams. However, before this data becomes useful, it typically requires cleaning, validation, and transformation before being loaded into a data warehouse for analytics, dashboards, machine learning, or any other use case you might have.

Knowing that, in this post, we'll walk through the basics of building a simple but effective ETL pipeline using AWS Lambda, Amazon S3, and Amazon RDS PostgreSQL. This post is not about creating the perfect and most optimized ETL solution but rather about demonstrating how AWS services can be used together to automate data processing in a straightforward way. By the end of this guide, you'll understand how to trigger a Lambda function to process data and store structured outputs in S3 for further analysis.

Github Repo: [click here](https://github.com/betaacid/etl-pipeline-aws-guide)

### What will we build?
For our example pipeline, we will take a JSON file containing recipe information, validate the data and extract the necessary information, transform the extracted data into clean and query-ready data and finally load it into a PostgreSQL database where it will be ready to be used for any use case you might have.

### Architecture Diagram
![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/cma16k3khesfm07k4g3rng44q)

### What will we use and why?

- **AWS Lambda:** We will use Lambda Functions to do the heavy lifting of extracting, transforming and loading the data. We chose Lambda especially for the serverless nature of the service, this way we don't need to worry about the underlying infrastructure and can focus on the code we want to execute. It also helps us that Lambdas are event-driven, so we can trigger the function from a variety of sources, like S3 events, API Gateway, or even a cron job. This resource is also very cheap to run, adopting a pay-per-use model.
- **Amazon S3:** We will use S3 to store the raw data, the temporary data, and the processed data. S3 is a very cheap and flexible object storage service that is well integrated with the rest of the AWS ecosystem. S3 also has a feature called S3 Event Notifications, that allows us to trigger a Lambda function when an object is created in a bucket. This is very useful for our use case, as we can trigger the Lambda function whenever a new file is uploaded to the S3 bucket.
- **Amazon RDS PostgreSQL:** RDS is a managed PostgreSQL database service that simplifies querying through standard SQL while maintaining good performance for frequent access patterns. RDS is also a managed service, which handles backups, updates, and other maintenance tasks, this way we can focus on the code we want to execute. Other viable options would be to use Amazon Aurora or Amazon Redshift, or even Redshift Spectrum if you want to build a solid data warehouse, but for this use case I felt that setting up Amazon Glue or any other metadata store would be overkill.
- **AWS SAM:** SAM is a tool that allows us to define AWS resources in a declarative way. Here we will use it to create the necessary resources for our pipeline.

### Let's get started
**Prerequisites**
- AWS Account (I will be using the free tier)
- AWS CLI
- Python 3.x
- AWS SAM CLI


**The template**

The SAM template is the main file that defines the resources for our pipeline. It's a YAML file that describes the AWS resources and their properties. To start, we will create the structure of the template.

```
AWSTemplateFormatVersion: "2010-09-09"
Description: ETL Pipeline for Recipe Processing
Transform: AWS::Serverless-2016-10-31

Parameters:
  Environment:
    Type: String
    Default: dev

  RDSPostgreSQLHost:
    Type: String
    Default: localhost

  RDSPostgreSQLPort:
    Type: String
    Default: "5432"

  RDSPostgreSQLDBName:
    Type: String
    Default: postgres

  RDSPostgreSQLUsername:
    Type: String
    Default: postgres

  RDSPostgreSQLPassword:
    Type: String
    Default: postgres

  RDSPostgreSQLSchema:
    Type: String
    Default: public

  RDSPostgreSQLTable:
    Type: String
    Default: recipes

Resources: ...
```

This is a good starting point, it defines the environment variable that we will use to deploy the template to different environments, but it's still empty. We need to add the resources here for it to be useful.

### Step 1: Creating the necessary IAM roles
First, we need to create a Lambda execution role that will allow the Lambda functions to access the necessary resources.

```
Resources:
  ...

  LambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
            Action: sts:AssumeRole
      Policies:
        - PolicyName: LambdaExecutionPolicy
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: Allow
                Action:
                  - s3:GetObject
                  - s3:PutObject
                  - s3:ListBucket
                  - logs:CreateLogGroup
                  - logs:CreateLogStream
                  - logs:PutLogEvents
                Resource: "*"
```

In this step we are defining the policy that will allow the Lambda functions to access the necessary resources. We are allowing the Lambda functions to get, put, and list objects in the S3 buckets as well as to create log groups and log streams in CloudWatch.

### Step 2: Create the S3 buckets
Now that we have the necessary IAM roles, we can create the S3 buckets that will store the raw data, the temporary data, the failed data, and the processed data.

```
Resources:
  ... 

  RawRecipesBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Sub "raw-recipes-data-${Environment}"
      AccessControl: Private

  TempRecipesBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Sub "temp-recipes-bucket-${Environment}"
      AccessControl: Private

  FailedRecipesBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Sub "failed-recipes-bucket-${Environment}"
      AccessControl: Private

  ProcessedRecipesBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Sub "processed-recipes-bucket-${Environment}"
      AccessControl: Private
```

Here we define the 4 buckets that we will use in our pipeline. We will use the !Sub intrinsic function to dynamically create the bucket names using the Environment parameter defined in the Parameters section. Also, we set the AccessControl property to Private, so no one can access the buckets without the proper permissions.

### Step 3: Create the Lambda functions
With the buckets and IAM roles created we can proceed and create our Lambda functions. In this case we will create 3 Lambda functions:

extraction-lambda: Where we will extract the data from the raw data
transformation-lambda: Where we will transform the data
loader-lambda: Where we will load the data into the RDS PostgreSQL database
...

```
Resources:
  ... # Our recently created buckets

  DataExtractionLambda:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: !Sub "extraction-lambda-${Environment}"
      Handler: data_extraction.lambda_handler
      Runtime: python3.9
      CodeUri: lambdas/data_extraction/
      Role: !GetAtt LambdaExecutionRole.Arn
      Environment:
        Variables:
          RAW_BUCKET: !Ref RawRecipesBucket
          TEMP_BUCKET: !Ref TempRecipesBucket
          FAILED_BUCKET: !Ref FailedRecipesBucket
      Timeout: 300
      MemorySize: 512

  DataTransformationLambda:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: !Sub "transformation-lambda-${Environment}"
      Handler: data_transformation.lambda_handler
      Runtime: python3.9
      CodeUri: lambdas/data_transformation/
      Role: !GetAtt LambdaExecutionRole.Arn
      Environment:
        Variables:
          TEMP_BUCKET: !Ref TempRecipesBucket
          PROCESSED_BUCKET: !Ref ProcessedRecipesBucket
      Timeout: 300
      MemorySize: 512

  DataLoaderLambda:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: !Sub "loader-lambda-${Environment}"
      Handler: data_loader.lambda_handler
      Runtime: python3.9
      CodeUri: lambdas/data_loader/
      Role: !GetAtt LambdaExecutionRole.Arn
      Environment:
        Variables:
          RDS_HOST: !Ref RDSPostgreSQLHost
          RDS_PORT: !Ref RDSPostgreSQLPort
          RDS_DB_NAME: !Ref RDSPostgreSQLDBName
          RDS_USERNAME: !Ref RDSPostgreSQLUsername
          RDS_PASSWORD: !Ref RDSPostgreSQLPassword
          RDS_SCHEMA: !Ref RDSPostgreSQLSchema
          RDS_TABLE: !Ref RDSPostgreSQLTable
          PROCESSED_BUCKET: !Ref ProcessedRecipesBucket
      Timeout: 300
      MemorySize: 512

  # Create permissions for S3 buckets to invoke Lambda functions
  RawBucketPermission:
    Type: AWS::Lambda::Permission
    Properties:
      Action: lambda:InvokeFunction
      FunctionName: !Ref DataExtractionLambda
      Principal: s3.amazonaws.com
      SourceArn: !GetAtt RawRecipesBucket.Arn

  TempBucketPermission:
    Type: AWS::Lambda::Permission
    Properties:
      Action: lambda:InvokeFunction
      FunctionName: !Ref DataTransformationLambda
      Principal: s3.amazonaws.com
      SourceArn: !GetAtt TempRecipesBucket.Arn

  ProcessedBucketPermission:
    Type: AWS::Lambda::Permission
    Properties:
      Action: lambda:InvokeFunction
      FunctionName: !Ref DataLoaderLambda
      Principal: s3.amazonaws.com
      SourceArn: !GetAtt ProcessedRecipesBucket.Arn
```

Here we define the 3 Lambda functions and the notification function that we will use in our pipeline. We will use the !GetAtt LambdaExecutionRole.Arn intrinsic function to get the ARN of the Lambda execution role that we will create in the next step.

We are also defining the events that will trigger the Lambda functions. The DataExtractionTrigger will trigger the DataExtractionLambda when a new object is created in the RawRecipesBucket. The DataTransformationTrigger will trigger the DataTransformationLambda when a new object is created in the TempRecipesBucket. The DataLoaderTrigger will trigger the DataLoaderLambda when a new object is created in the ProcessedRecipesBucket.

The CodeUri property is the path to the Lambda function code in the lambdas folder that we will create soon.

### Step 4: Create the notification function

```yaml
Resources:
  ...

  # Create notifications through CloudFormation custom resources
  NotificationFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: notification_handler.lambda_handler
      Role: !GetAtt NotificationFunctionRole.Arn
      Runtime: python3.9
      Timeout: 30
      CodeUri: lambdas/notification_handler/

  # Custom resource Lambda role for S3 notifications
  NotificationFunctionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
      Policies:
        - PolicyName: S3BucketNotificationPolicy
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - s3:PutBucketNotification
                  - s3:GetBucketNotification
                  - s3:PutBucketNotificationConfiguration
                Resource: '*'

  # S3 bucket notification configurations
  RawBucketConfiguration:
    Type: Custom::S3BucketNotification
    DependsOn: 
      - RawBucketPermission
      - DataExtractionLambda
    Properties:
      ServiceToken: !GetAtt NotificationFunction.Arn
      BucketName: !Ref RawRecipesBucket
      NotificationConfiguration:
        LambdaFunctionConfigurations:
          - Events: ["s3:ObjectCreated:*"]
            LambdaFunctionArn: !GetAtt DataExtractionLambda.Arn

  TempBucketConfiguration:
    Type: Custom::S3BucketNotification
    DependsOn: 
      - TempBucketPermission
      - DataTransformationLambda
    Properties:
      ServiceToken: !GetAtt NotificationFunction.Arn
      BucketName: !Ref TempRecipesBucket
      NotificationConfiguration:
        LambdaFunctionConfigurations:
          - Events: ["s3:ObjectCreated:*"]
            LambdaFunctionArn: !GetAtt DataTransformationLambda.Arn

  ProcessedBucketConfiguration:
    Type: Custom::S3BucketNotification
    DependsOn: 
      - ProcessedBucketPermission
      - DataLoaderLambda
    Properties:
      ServiceToken: !GetAtt NotificationFunction.Arn
      BucketName: !Ref ProcessedRecipesBucket
      NotificationConfiguration:
        LambdaFunctionConfigurations:
          - Events: ["s3:ObjectCreated:*"]
            LambdaFunctionArn: !GetAtt DataLoaderLambda.Arn
```

The NotificationFunction is a CloudFormation custom resource used here to handle the S3 bucket notifications during the stack deployment.

The NotificationFunctionRole is the IAM role that will be used to trigger the Lambda functions.

The RawBucketConfiguration, TempBucketConfiguration, and ProcessedBucketConfiguration are the custom resources that will be used to trigger the Lambda functions when a new object is created in the S3 buckets.

Note: This Function was added because when using native S3 Event Notifications alone with CloudFormation, the deployment process Failed due to circular dependencies between the S3 buckets and the Lambda functions. Thanks to [wjordan](https://stackoverflow.com/a/45022208) for the workaround.

### Step 5: Create the RDS PostgreSQL database
Now that we have the Lambda functions created, we can proceed and create the RDS PostgreSQL database.

```
Resources:
  ...

  RDSPostgreSQL:
    Type: AWS::RDS::DBInstance
    Properties:
      DBInstanceIdentifier: !Sub "rds-postgresql-${Environment}"
      Engine: postgres
      EngineVersion: "16.8"
      DBParameterGroupName: !Ref RDSPostgreSQLParameterGroup
      DBInstanceClass: db.t3.micro
      DBName: !Ref RDSPostgreSQLDBName
      MasterUsername: !Ref RDSPostgreSQLUsername
      MasterUserPassword: !Ref RDSPostgreSQLPassword
      AllocatedStorage: 20
      StorageType: gp2
      StorageEncrypted: true
      MaxAllocatedStorage: 20
      PubliclyAccessible: true
      DeletionProtection: false
```

Here we define the RDS PostgreSQL database that we will use in our pipeline. We are using the !Ref RDSPostgreSQLParameterGroup intrinsic function to get the parameter group that we will create in the next step.

### Step 6: Create the parameter group
Now that we have the RDS PostgreSQL database created, we can proceed and create the parameter group.

```
Resources:
  ...

  RDSPostgreSQLParameterGroup:
    Type: AWS::RDS::DBParameterGroup
    Properties:
      DBParameterGroupName: !Sub "rds-postgresql-parameter-group-${Environment}"
      Family: postgres15
      Description: Parameter group for RDS PostgreSQL
      Parameters:
        max_connections: 100
        shared_preload_libraries: pg_cron
        cron.database_name: postgres
```

Here we define the parameter group that we will use in our RDS PostgreSQL database. We are using the !Sub intrinsic function to dynamically create the parameter group name using the Environment parameter defined in the Parameters section.

### Step 7: Putting it all together
Now that we have all the components in place, here's how the complete pipeline works:

1. Raw JSON files are uploaded to the raw-recipes-bucket
2. The S3 event triggers the extraction Lambda function via the custom notification resource
3. The extraction Lambda:
     - Validates the data
     - Extracts relevant fields
     - Stores valid data in the temp-recipes-bucket
     - Stores invalid data in the failed-recipes-bucket
4. The transformation Lambda is triggered by new files in the temp-recipes-bucket
5. The transformation Lambda:
     - Transforms the data into the final format
     - Stores processed data in the processed-recipes-bucket
6. The loader Lambda is triggered by new files in the processed-recipes-bucket
7. The loader Lambda:
     - Loads the data into the RDS PostgreSQL database

### Next Steps
To enhance this pipeline, consider adding:
1. Error handling and retry mechanisms
2. Data quality checks and metrics
3. Partitioning of the processed data in S3
4. Monitoring and alerting for pipeline failures
5. Data versioning and lineage tracking
6. Adding more complex data tools like AWS Glue, AWS Athena, AWS Redshift, etc.

### Conclusion
This post covered the basics of building a serverless ETL pipeline using AWS Lambda, S3, and RDS PostgreSQL. We walked through the process of creating the necessary IAM roles, S3 buckets, Lambda functions, and RDS PostgreSQL database. Although we faced some issues with the S3 Event Notifications that required us to create some workarounds, overall, this was a fun project to work on it shows us how we can simplify our life as software engineers by using existing tools and services in available in the AWS ecosystem in our favor to build scalable and efficient systems, with minor headaches in some of the integrations, but overall it was a great experience to be able to explore these resources.

A Hands-On Guide to Serverless ETL with AWS

Adam Smith

Back in 2016, Apple introduced [Siri Intents](https://developer.apple.com/documentation/sirikit/)—a framework that allowed developers to expose their app’s functionality to Siri. If you’ve ever asked your iPhone to “send a message,” “order coffee,” or “start a workout,” you’ve probably used them. Siri wasn’t just listening; it was matching your voice command to a structured interface developers had defined.

Fast forward to today, and we’re seeing a similar—but broader—concept emerge in the world of language models: the [Model Context Protocol](https://www.anthropic.com/news/model-context-protocol) **(MCP)** from Anthropic. It's a new open protocol aimed at helping AI agents (like Claude or ChatGPT) interface with external tools and APIs in a secure, declarative, and dynamic way.

On the surface, these systems feel worlds apart—one is Apple's voice assistant; the other is an interface layer for LLMs. But conceptually? They're siblings.

### Both Were Built to Answer the Same Question

*“How can we give a smart system safe, predictable access to tools it doesn’t own?”*

For Siri Intents, the goal was to make Siri useful beyond built-in apps. Apple needed a way to say: Here’s how to let Siri send a Venmo payment or start your Peloton workout—without guessing how those apps work under the hood.

For MCP, the goal is nearly identical—but for LLMs. Large models are great at generating language, but they’re not inherently wired into your calendar, CRM, or shipping database. MCP lets models discover and invoke those external tools in a structured way—no guessing, no scraping, no brittle plugins.

### How Siri Intents Work

Siri Intents require developers to define their app’s capabilities using .intentdefinition files. These described:

- The *intent* name (“SendPayment”)
- The *parameters* it accepted (“amount”, “recipient”)
- And the *expected* response (“paymentSuccess”, “errorMessage”)



This structure allowed Siri to match user queries to real-world actions. It also let Apple maintain a tight control loop: developers declared what they supported, users got autocomplete-style suggestions, and the assistant never went off-script.
That predictability is what made the system safe, even if it sometimes felt limiting.

### MCP Takes That Idea and Cranks It to Eleven

MCP plays a similar role—but for general-purpose models, not voice assistants.
Instead of `.intentdefinition` files, you expose tools described with JSON schemas or OpenAPI specs. Each tool defines:

- A name and description
- The parameters it accepts
- And the expected output



The model—Claude, for instance—can “see” this information in its context window and decide if/when to call a tool. This opens the door to AI that not only understands natural language but executes actions based on it—fetching data, triggering workflows, or chaining multiple tools together.

MCP is still in its early stages. While the core spec is available and Anthropic is already experimenting with it inside Claude, it's a living standard. There’s no official governance body yet, and many key design decisions—especially around authentication, rate-limiting, and permissions—are still being debated in public.

### Siri’s Walled Garden vs. MCP’s Open Frontier

One major difference between the two systems is **security**. Siri Intents live in a tightly controlled Apple ecosystem. Apps are vetted, permissions are managed through the OS, and there’s a clear trust model. The assistant only does what it’s explicitly allowed to do—and only with approved apps.

MCP, on the other hand, operates in a much looser, open-world context. Any developer can expose a tool. Any model that supports MCP can potentially invoke it.

That’s powerful—but also dangerous.

As Elena Cross put it in her excellent write-up, “[🛡️ The ‘S’ in MCP Stands for Security,](https://elenacross7.medium.com/%EF%B8%8F-the-s-in-mcp-stands-for-security-91407b33ed6b)” the core challenge ahead for MCP is **ensuring trust, authentication, and context-aware permissions in a world where models might call anything**.

There’s no OS-level sandbox like iOS. Which means things like OAuth, scope-limited access, and human-in-the-loop verification need to be first-class citizens of any serious MCP implementation.

### Where They're Alike

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/cm9a515cl1c5a07ir5infea22)

Both aim to solve the same core problem: **How can we create a safe, structured, and extensible interface between an assistant and the outside world?**

### Why It Matters

MCP could become the glue that binds future AI systems to the real world.

Anything Siri Intents could do—like ordering a pizza, sending a message, or starting a workout—MCP can do too. But it doesn’t stop there.
Want a model to book a meeting? MCP gives it structured access to your calendar API.

Want it to browse a knowledge base, summarize insights, and kick off a ticket in your support system? MCP enables that entire chain—because the tools are composable, and the agent is smart enough to use them.

Where Siri Intents gave voice assistants a limited but useful playbook, MCP hands AI models a full toolkit—and the ability to reason about when and how to use each tool, in context.

### Final Thought

When Siri Intents launched, it was ahead of its time. It brought predictability and composability to the messy world of natural language. But it was locked in a walled garden.

MCP builds on that idea—and takes it cross-platform, cross-service, and model-native. You don’t need to be on iOS. You just need to have a tool worth calling.

But with great power comes real risk. If MCP is going to succeed, it’ll need more than a spec. It’ll need a trust framework that lets developers and users decide what models can do, when, and on whose behalf.

Siri Intents Walked So Model Context Protocol Could Run

Daniel Moura

**Proposal:**[https://github.com/tc39/proposal-is-error](https://github.com/tc39/proposal-is-error)

**Stage:** 3

In our previous post, we examined the [Atomics.pause](https://betaacid.co/blog/boosting-multithreaded-performance-with-atomicspause-in-javascript) feature and how it can improve asynchronous task management in JavaScript. Building on that discussion of enhancing the developer experience, we now turn to Error.isError, another emerging TC39 proposal that addresses a different yet equally significant challenge.

Error handling is a fundamental aspect of robust application development. However, identifying an Error object reliably becomes challenging in complex JavaScript environments involving multiple realms (like iframes, workers, or Node.js VM contexts). The Error.isError proposal introduces a straightforward and standard method to check if a value is an Error, regardless of the realm it originated from.

### **The Problem**

In JavaScript, the common way to check if a value is an Error is by using the instanceof operator:

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/cm8qdoia12wxo06k26uowzf80)

However, this method has limitations in environments with multiple realms. Each realm in JavaScript has its own set of built-in objects, including the Error constructor. An Error object created in one realm doesn't pass the instanceof Error check in another realm:

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/cm8qdqlul2y8v08iqncahjsl2)

This inconsistency arises because errorInA is an instance of RealmA.Error, not RealmB.Error. As a result, developers cannot reliably detect Error objects across realms, leading to unhandled errors and unpredictable behavior in applications that rely on cross-realm interactions.

Moreover, with the introduction of Symbol.toStringTag, methods like Object.prototype.toString are no longer reliable for brand checking, as the toString result can be spoofed.

### **Syntax and Examples**

To address these issues, the Error.isError proposal introduces a static method that provides a reliable way to determine if a given value is an Error object, regardless of the realm or any Symbol.toStringTag spoofing.

**Syntax**

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/cm8qds2792xbs07k3j0e3aio2)

This method returns true if value is an instance of Error or any of its subclasses, even if it comes from a different realm.

**Example**

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/cm8qdt42t2xus07k30zat9s9i)

The method checks for the presence of the internal [[ErrorData]] slot, which is a part of the specification for Error objects and their subclasses. This internal slot is not accessible or spoofable from JavaScript code, ensuring the reliability of the check.

### **Use Cases**

**Cross-Realm Error Handling**

In applications that use iframes, web workers, or other multi-realm setups, errors can originate from different realms. Using Error.isError ensures that these errors are correctly identified and handled.

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/cm8qduooq2ye407k3hqxqrk44)

**Library Development**

Library authors often need to handle errors that could originate from various environments. Using Error.isError allows libraries to robustly detect and handle errors without worrying about the realm in which they were created.

**Serialization and Structured Cloning**

Platforms that need to serialize values safely, such as certain server environments or tools like RunKit, can use Error.isError to identify Error objects accurately before serialization.

In the context of the structuredClone method—which has special behavior for Error objects—developers can predict whether the special handling will be applied.

### **Potential Pitfalls**

While Error.isError offers a more reliable way to detect Error objects, there are some considerations:

**Not Yet Standardized:** 
As of now, Error.isError is still a proposal. It is not available in all JavaScript environments. Developers should check for support or consider polyfills.

**Custom Error Objects:** 
Objects that mimic Errors but don't inherit from the standard Error prototype chain won't be recognized by Error.isError. This method checks for the presence of the internal [[ErrorData]] slot, which custom error-like objects might not have.

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/cm8qdwfdr2zah07k3djyvw84n)

### **Conclusion**

The Error.isError proposal represents a significant advancement in improving error handling reliability and consistency in JavaScript, particularly for applications that operate across multiple realms. By introducing a standardized method to identify Error objects—including subclasses and cross-realm instances—it streamlines error detection and management across diverse execution contexts. This proposal addresses a long-standing challenge, making it easier for developers to handle errors robustly and consistently in complex environments.

Detecting Errors Reliably: A Deep Dive into the Error.isError Proposal

Gabriel Valerio

As a developer working with cloud services, I've found that serverless computing makes my life much easier. Instead of managing servers, I can focus on writing code that matters. Let me share my experience with Azure Functions and AWS Lambda - two popular serverless platforms I use regularly.

### Azure Functions: Microsoft's Serverless Offering

Azure Functions is Microsoft's offering in the serverless space. It's designed to be easy to use, especially if you're already familiar with Microsoft's ecosystem. Setting up an Azure Function is relatively straightforward. With tools like Visual Studio Code, you can create, debug, and deploy functions quickly.

Here's an example of an Azure Function built to process image uploads:

```typescript
import { AzureFunction, Context, HttpRequest } from "@azure/functions";
import sharp from "sharp";

const imageProcessor: AzureFunction = async function (context: Context, req: HttpRequest): Promise<void> {
    const imageBuffer = Buffer.from(req.body, 'base64');
    
    try {
        // Resize the image to a thumbnail
        const thumbnail = await sharp(imageBuffer)
            .resize(200, 200, { fit: 'cover' })
            .jpeg({ quality: 80 })
            .toBuffer();
            
        context.res = {
            status: 200,
            body: thumbnail.toString('base64'),
            headers: {
                'Content-Type': 'image/jpeg'
            }
        };
    } catch (error) {
        context.res = {
            status: 400,
            body: "Failed to process image"
        };
    }
};
```

This function takes an uploaded image, creates a thumbnail version, and sends it back. Just a few lines of code, and you have a fully functional endpoint.

### AWS Lambda: Amazon's Serverless Service

AWS Lambda is Amazon's serverless offering. It's been around longer and has a more mature ecosystem. Lambda functions are event-driven, meaning they can be triggered by a variety of events, from HTTP requests to changes in an S3 bucket.

Here's an example Lambda function that helps moderate user comments:

```typescript
import { DynamoDB } from 'aws-sdk';
import { ComprehendClient, DetectSentimentCommand } from '@aws-sdk/client-comprehend';

export const handler = async (event: any): Promise<any> => {
    const comment = event.body?.comment;
    if (!comment) {
        return {
            statusCode: 400,
            body: JSON.stringify({ error: 'Comment is required' })
        };
    }

    const comprehend = new ComprehendClient({ region: 'us-east-1' });
    
    try {
        // Analyze comment sentiment
        const sentiment = await comprehend.send(new DetectSentimentCommand({
            Text: comment,
            LanguageCode: 'en'
        }));
        
        // Store comments that aren't negative
        if (sentiment.Sentiment !== 'NEGATIVE') {
            const dynamo = new DynamoDB.DocumentClient();
            await dynamo.put({
                TableName: 'Comments',
                Item: {
                    id: Date.now().toString(),
                    comment,
                    sentiment: sentiment.Sentiment
                }
            }).promise();
        }

        return {
            statusCode: 200,
            body: JSON.stringify({
                approved: sentiment.Sentiment !== 'NEGATIVE',
                sentiment: sentiment.Sentiment
            })
        };
    } catch (error) {
        console.error('Error:', error);
        return {
            statusCode: 500,
            body: JSON.stringify({ error: 'Failed to process comment' })
        };
    }
};
```

This function analyzes incoming comments using Amazon Comprehend to detect negative content, storing only appropriate comments in a database. It's a super helpful way of automating moderation quickly and cheaply.

### Which One Should You Choose?

Both services are solid choices, but here's what I've found:

Azure Functions works best when:

- You're already using other Microsoft services like Azure SQL, Azure AI services, etc.
- You need tight integration with Azure DevOps
- You want built-in dependency injection and middleware support
- Azure provides a set of tools that you can use such as a CLI or even easier, a VS Code extension to simplify and automate your deployments.

AWS Lambda shines when:

- You need a wide variety of event triggers
- You want to use Amazon's AI services such as SQS, S3, Step Functions, etc.
- Cost is a major factor (Lambda's pricing can be more granular)
- AWS provides a great CLI to work with CloudFormation templates and automate your deployment process: sam.

Ultimately, the choice between Azure Functions and AWS Lambda will depend on your specific needs and preferences. If you're deeply integrated with Microsoft services, Azure Functions might be the better choice. If you're already using AWS, then AWS Lambda would be more seamless to adopt.

### Final Thoughts

At Beta Acid, we understand that every project is unique, and what works for one client might not work for another. That's why we're committed to working with the best tools for the job, whether that's Azure Functions, AWS Lambda, or another technology entirely. Our goal is to deliver solutions that meet our clients' needs effectively and efficiently.

If you're considering serverless computing for your next project, we'd love to hear from you. Whether you're leaning towards Azure Functions or AWS Lambda, or if you're still deciding, we're here to help you navigate the options and find the best fit for your needs. Let's work together to bring your vision to life.



Exploring Azure Functions and AWS Lambda

Thiago Afonso

# React Native Performance Solutions: Debug Like a Pro

Debugging: the art of being a detective in a crime scene where you're also the prime suspect. While traditional debugging tools help you find out why your perfectly written code (you’d think!) decided to throw a tantrum, certain tools are specifically designed to provide insights into performance bottlenecks, enabling developers to improve the efficiency of their applications.

Performance in React Native isn’t exactly a cakewalk. It’s a delicate balancing act between JavaScript, native modules, and the occasional head-scratcher. To achieve smooth performance, you first need to know where the bottlenecks are—and that’s where debugging tools come in. Let’s dive in!

## Traditional Debugging Tools

React Native offers several built-in tools to assist in the debugging process:

- **In-App Developer Menu**: Shake your phone like a Polaroid picture (or use [keyboard shortcuts](https://reactnative.dev/docs/debugging#opening-the-dev-menu)) to open the developer menu. You’ll find options like hot reloading, element inspection, and a dash of performance profiling.

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/5s8pCtjzRbWAXWT9ahcc)

- **Chrome DevTools**: Connect to your app and debug JavaScript execution, monitor network requests, and keep an eye on every moving part.

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/wxXNrjhFQJCDTDtTDc6Y)
    
These are fine tools, but they often leave you wondering about what’s slowing things down. Let’s level up.

# Performance-Focused Debugging Tools

When it comes to optimizing React Native applications, having the right tools is crucial. These performance-focused debugging tools go beyond basic error tracking—they help you understand the inner workings of your app, identify performance bottlenecks, and make data-driven optimization decisions. Let's explore some of the most powerful tools in a React Native developer's performance optimization toolkit.

## **1 - Why Did You Render (WDYR)**

WDYR is like a lie detector test for React components—it tells you exactly when and why they re-render. Imagine spotting that one child component re-rendering 500 times because of a single prop change. With WDYR, you can catch and fix these “oops” moments.

Its installation process is very straightforward and simple. Check it out:

- Install the package:
    
    ```bash
    npm install @welldone-software/why-did-you-render --save-dev
    ```
    

- Create a `wdyr.js` file in the root directory with the following content:
    
    ```jsx
    import React from 'react';
    import whyDidYouRender from '@welldone-software/why-did-you-render';
    
    if (process.env.NODE_ENV === 'development') {
      whyDidYouRender(React, {
        trackAllPureComponents: true,
      });
    }
    ```
    

- Import `wdyr.js` at the top of your entry file (e.g., `index.js`):
    
    ```jsx
    import './wdyr';
    ```
    

Now, your console will provide detailed logs showing which components are re-rendering and why. You'll see information about prop changes, state updates, and other triggers that cause components to refresh unnecessarily. 

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/eFx1MdvXQzml4afsfe2D)

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/UMWaNViQDeiYqVUmC09g)

---

## **2 - React Native DevTools**

Meet the shiny new [React Native DevTools](https://reactnative.dev/docs/react-native-devtool)—a step up from Chrome DevTools. It combines React Inspector, Redux DevTools, and a profiler to track performance issues and visualize bottlenecks.

![debugging-rndt-sources-paused-with-device-d1d48a3df5a69d3bf92a16845f0f9c12.jpg](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/lxOUMiZWR2inylFuPkYF)

**Why It’s Awesome:**

- **Profiler:** Monitor component renders and check why specific components are slow.
- **Component Tree:** Understand the hierarchy and pinpoint which elements are overworking themselves.
- **Redux Insights:** Track state changes for apps using Redux.

Here's how to quickly enable it:

- Upgrade React Native to version 0.70 or later.
- Open the DevTools by running `npx react-devtools`.

With these tools, you can identify exactly which components need optimization and improve your app’s performance.

---

# **Why Optimize?**

Performance optimization isn't just a technical necessity—it's the difference between an app that delights users and one that frustrates them. Slow load times, laggy interactions, and unresponsive interfaces can turn even the most promising app into a source of user frustration. 

When your app runs at a snail’s pace, this is how your users feel:

![](https://media.giphy.com/media/xFLDPwNQt5gOY/giphy.gif?cid=ecf05e47ihsptyo2511yke5pt6ltabryb8wz1sdvfcvek9db&ep=v1_gifs_related&rid=giphy.gif&ct=g)

*Don’t let your app be the reason phones meet walls.*

## Implementing Performance Optimizations

Once performance bottlenecks are identified using the tools above, developers can apply React best practices to optimize their applications:

- **Memoization**: Utilize `React.memo` to prevent unnecessary re-renders of functional components when their props haven't changed.
- **useCallback and useMemo Hooks**: These hooks help memoize functions and values, ensuring that components do not re-render due to unchanged dependencies.
- **Pure Components**: For class components, extending `React.PureComponent` enables shallow prop and state comparison, reducing unnecessary updates.
- **Optimize Lists with FlatList**: When rendering large lists, use `FlatList` instead of `ScrollView` to improve performance by rendering only visible items.
- **Avoid Inline Functions**: Define functions outside of render methods to prevent re-creation on every render, which can lead to unnecessary re-renders of child components.

## Conclusion

Debugging in React Native isn’t just about fixing bugs; it’s about building apps that make users smile instead of rage quit. By using tools like WDYR and the new React Native DevTools, you can identify performance bottlenecks and create buttery-smooth apps.

So, put on your detective hat, squash those bugs, and make your app a joy to use. Your users—and their phones—will thank you.

React Native Performance Solutions: Debug Like a Pro

**Proposal:** [https://github.com/tc39/proposal-atomics-microwait](https://github.com/tc39/proposal-atomics-microwait)

**Stage:** 3

In our previous post, [Simplifying Array Combinations with Array.zip and Array.zipKeyed](https://betaacid.co/blog/simplifying-array-combinations-with-arrayzip-and-arrayzipkeyed), we explored how JavaScript's evolving features make data manipulation more intuitive. Now, let's examine another exciting development: the `Atomics.pause` proposal—a feature designed to optimize concurrent and multithreaded JavaScript applications.

As applications become more complex and performance-critical, efficient concurrency is crucial. Led by [Shu-yu Guo](https://github.com/syg) and now at **Stage 3** in the TC39 process, `Atomics.pause` introduces micro-waiting to JavaScript. This feature enhances how threads handle wait conditions, boosting performance and resource usage in both high-performance systems and resource-limited environments.

## The Problem

JavaScript's concurrency is limited by not having good ways for threads to wait briefly or monitor condition changes. Developers resort to busy-wait loops that constantly check conditions, which wastes CPU resources and hurts performance. This creates significant problems for high-performance applications and resource-limited devices.

Additionally, implementing efficient spin locks—a common concurrency control mechanism—needs low-level CPU instructions that JavaScript can't fully utilize. For instance, spin loops typically require CPU hints like the `pause` instruction on x86 architectures to help sibling cores share resources effectively.

### Micro Waits in JavaScript

The concept of micro-waiting involves threads pausing execution for very short periods, using CPU-level instructions to yield shared resources without relinquishing control of the core. This is different from traditional thread yielding, which is managed by the operating system and involves more overhead.

**Motivation**

An efficient lock acquisition in concurrent programming typically follows this pattern:

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/xRL38diXS8KAnk4cyoyx)

In this algorithm:

- **Fast Path:** The thread attempts to acquire the lock and, if unsuccessful, spins for a short time using `SpinForALittleBit()`. This is efficient when contention is low.
- **Slow Path:** If the lock isn't acquired after a certain number of spins, the thread goes to sleep using `PutThreadToSleepUntilLockReleased()`, which is more efficient when contention is high.

Implementing `SpinForALittleBit()` optimally in JavaScript is challenging because it requires CPU hinting instructions that are not currently accessible. Similarly, putting a thread to sleep efficiently is problematic, especially on the main thread where blocking is disallowed to prevent UI hangs.

## Syntax and Examples

The proposed `Atomics.pause` function allows a thread to pause execution until a condition changes or a timeout occurs.

### Syntax

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/hQpe04NER8SYncsU2cJl)

### Parameters

- `N:` A non-negative integer controlling the pause duration. Larger values of `N` result in longer pauses.

The `Atomics.pause` method allows a thread to perform a finite-time wait, efficiently pausing execution without consuming excessive CPU resources. It's important to note that this method does not block the thread or yield execution to another thread; it simply hints to the CPU to optimize resource sharing.

### Example

Here's how you might use Atomics.pause in a spin lock implementation:

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/Kt5U8lQsSjOPqG2BJ2UV)

In this example, `Atomics.pause(spins)` introduces a micro-wait that increases with each iteration, implementing an exponential backoff strategy.

## Use Cases

**High-Performance Computing:** Applications that require tight synchronization between threads, such as simulations or real-time data processing, can benefit from `Atomics.pause` by reducing overhead and improving efficiency.

**Resource-Constrained Devices:** Avoiding busy-wait loops on devices with limited CPU resources can save battery life and improve performance.

**Gaming:** Game engines and real-time applications that require precise timing and synchronization can use micro-waiting to improve frame rates and responsiveness.

## Potential Pitfalls

**Browser Support:** As a Stage 3 proposal, `Atomics.pause` may not be available in all JavaScript environments. Developers should ensure compatibility or provide fallbacks when using this feature.

**Misuse Leading to Deadlocks:** Improper use of `Atomics.pause`, such as not ensuring a condition will eventually become true, could lead to deadlocks. It's crucial to design your synchronization logic carefully.

**Main Thread Blocking:** While `Atomics.pause` does not block the main thread, attempting to implement blocking behavior on the main thread is discouraged due to potential UI hangs and responsiveness issues.

## Conclusion

The `Atomics.pause` proposal marks a crucial step forward in JavaScript's concurrency model. By enabling efficient micro-waiting, it empowers developers to create high-performance multithreaded code without relying on wasteful busy-wait loops.

Boosting Multithreaded Performance with Atomics.pause in JavaScript

Otavio Barros

Choosing the right web framework can significantly impact your project's success. Next.js has emerged as a powerhouse in the React ecosystem, offering a robust solution for building modern web applications. As we explore Next.js 15, let's understand why developers and businesses increasingly rely on this framework.

In this post we'll explore significant improvements in Next’s routing, server components, caching behavior, and performance optimizations. These updates continue to reshape how we approach modern web development, making it easier to build faster, more reliable applications.

Let's start by examining the revolutionary App Router, followed by deep dives into server components, caching strategies, and solutions to common performance challenges.

### The Power of App Router

Next.js 15's App Router represents a new approach to application structure, building upon the stable foundation introduced in version 13. At its core, the App Router uses a file-system based routing that transforms how we organize and build web applications.

### Root Layout Structure

The foundation of any Next.js application starts with a root layout:

```
// app/layout.tsx
export default function RootLayout({
  children,
}: {
  children: React.ReactNode;
}) {
  return (
    <html lang="en">
      <body>
        <header>Global Navigation</header>
        <main>{children}</main>
        <footer>Global Footer</footer>
      </body>
    </html>
  );
}
```

This root layout wraps every page in your application, providing consistent UI elements across all routes.

### Intuitive Folder Structure

The App Router follows a predictable folder structure where each folder represents a route segment:

```
app/
├── layout.tsx
├── page.tsx
├── dashboard/
│   ├── layout.tsx
│   ├── page.tsx
│   └── settings/
│       └── page.tsx
└── blog/
    ├── layout.tsx
    └── [slug]/
        └── page.tsx

```

Each `page.tsx` file automatically becomes a route, while `layout.tsx` files create a shared UI for their respective segments.

### Nested Layouts

```
// app/dashboard/layout.tsx
export default function DashboardLayout({
  children,
}: {
  children: React.ReactNode;
}) {
  return (
    <div className="dashboard-container">
      <aside>
        <nav>Dashboard Navigation</nav>
      </aside>
      <section className="content">{children}</section>
    </div>
  );
}

```

This layout will wrap all pages within the dashboard section while preserving its state during navigation between dashboard routes.

### Special Files

The App Router introduces special files that handle common scenarios:

- `loading.tsx` for suspense boundaries
- `error.tsx` for error handling
- `not-found.tsx` for 404 cases
- `page.tsx` for route UI


```
// app/dashboard/loading.tsx
export default function Loading() {
  return <div className="dashboard-loader">Loading...</div>;
}
```

These files automatically create optimal loading and error states for your application.

The App Router's architecture naturally leads us to Server Components, which form the backbone of Next.js 15's performance optimizations. These components, rendered entirely on the server, work seamlessly with the App Router to deliver exceptional performance and developer experience.

### Server Components and Actions in Next.js 15

Next.js 15 enhances both Server Components and Server Actions, providing developers with powerful tools for building efficient, secure applications. Let's explore these features and their practical applications.

**Server Components**

Server Components fundamentally change how we build React applications by moving component rendering to the server. Here's a practical example:

```
// app/products/page.tsx
async function ProductList() {
  const products = await fetchProducts(); // Server-side data fetching

  return (
    <div className="grid">
      {products.map((product) => (
        <ProductCard
          key={product.id}
          product={product}
          // Sensitive data can be safely passed
          internalMetrics={product.metrics}
        />
      ))}
    </div>
  );
}
```
**Key Benefits:**

- Data fetching occurs closer to the data source
- Sensitive information remains server-side
- Reduced client-side JavaScript bundle
- Automatic caching and optimization

### Advanced Server Component Patterns

**Streaming with Suspense:**

```
// app/dashboard/page.tsx
import { Suspense } from "react";

export default function Dashboard() {
  return (
    <div>
      <Suspense fallback={<UserProfileSkeleton />}>
        <UserProfile />
      </Suspense>
      <Suspense fallback={<AnalyticsSkeleton />}>
        <AnalyticsChart />
      </Suspense>
    </div>
  );
}
```

**Parallel Data Fetching:**

```
// app/profile/page.tsx
async function ProfilePage() {
  // These requests run in parallel
  const [userData, userPosts, userAnalytics] = await Promise.all([
    fetchUserData(),
    fetchUserPosts(),
    fetchUserAnalytics(),
  ]);

  return (
    <div>
      <UserInfo data={userData} />
      <UserPosts posts={userPosts} />
      <UserStats analytics={userAnalytics} />
    </div>
  );
}

```

### Server Actions

Server Actions provide a secure way to handle server-side mutations. Next.js 15 introduces enhanced security features and better integration with forms.

**Basic Form Handling:**

```
// app/actions.ts
"use server";

export async function updateProfile(formData: FormData) {
  const name = formData.get("name");
  const email = formData.get("email");

  await db.user.update({
    where: { id: session.user.id },
    data: { name, email },
  });
}

// app/profile/page.tsx
export default function ProfileForm() {
  return (
    <form action={updateProfile}>
      <input name="name" type="text" />
      <input name="email" type="email" />
      <button type="submit">Update Profile</button>
    </form>
  );
}

```

**Advanced Server Action Pattern:**

```
// app/actions.ts
"use server";

export async function handleOrder(formData: FormData) {
  const items = JSON.parse(formData.get("items") as string);

  try {
    // Start transaction
    const order = await db.$transaction(async (tx) => {
      // Check inventory
      await validateInventory(items);
      // Process payment
      const payment = await processPayment(items);
      // Create order
      return await createOrder(items, payment);
    });

    revalidatePath("/orders");
    return { success: true, orderId: order.id };
  } catch (error) {
    return { success: false, error: error.message };
  }
}
```

**Client-Side Integration:**

```
"use client";

import { handleOrder } from "./actions";
import { useTransition } from "react";

export function CheckoutButton({ items }) {
  const [isPending, startTransition] = useTransition();

  const processCheckout = () => {
    startTransition(async () => {
      const formData = new FormData();
      formData.append("items", JSON.stringify(items));

      const result = await handleOrder(formData);
      if (result.success) {
        // Handle success
      }
    });
  };

  return (
    <button onClick={processCheckout} disabled={isPending}>
      {isPending ? "Processing..." : "Checkout"}
    </button>
  );
}
```

Server Components and Actions in Next.js 15 provide a robust foundation for building modern web applications. They enable developers to create performant, secure, and maintainable applications while keeping sensitive operations server-side. The next section will explore how Next.js 15 handles caching to further optimize application performance.

### Understanding Caching in Next.js 15

Next.js 15 introduces significant changes to its caching behavior, moving from an opinionated cached-by-default approach to giving developers more explicit control over caching strategies.

### Default Behavior

In version 15, requests are no longer cached by default. This provides more predictable behavior and better control over data freshness. Here's how to implement different caching strategies:

**Time-based Caching**

```
// Basic fetch with cache
async function getProducts() {
  const products = await fetch("https://api.store.com/products", {
    cache: "force-cache",
    next: { revalidate: 3600 }, // Revalidate every hour
  });

  return products.json();
}

// Using route segment config
export const revalidate = 3600; // Page-level revalidation

export default async function ProductsPage() {
  const products = await getProducts();
  return (
    <div className="products-grid">
      {products.map((product) => (
        <ProductCard key={product.id} {...product} />
      ))}
    </div>
  );
}

```

**On-Demand Revalidation:**

```
// app/actions.ts
"use server";

import { revalidatePath } from "next/cache";

export async function updateProduct(formData: FormData) {
  await saveProduct(formData);

  // Revalidate the products page and all its subpages
  revalidatePath("/products");
}

```

**Using Tag-based Revalidation:**

```
// Fetching with tags
async function getProduct(id: string) {
  const product = await fetch(`https://api.store.com/products/${id}`, {
    next: { tags: ["products"] },
  });
  return product.json();
}

// Revalidating tagged data
export async function updateProducts() {
  "use server";
  await updateProductInDatabase();
  revalidateTag("products");
}

```

### Request Deduplication

Next.js automatically deduplicates identical requests during the same render pass. Here's how to leverage this:

```

// This component can be used multiple times, but only one request will be made
async function ProductInfo({ id }: { id: string }) {
  const product = await fetch(`https://api.store.com/products/${id}`, {
    cache: "force-cache",
  });

  return product.json();
}

// Multiple instances will trigger only one request
export default async function Page() {
  return (
    <>
      <ProductInfo id="1" />
      <ProductInfo id="1" /> {/* Reuses cached data */}
      <OtherComponent>
        <ProductInfo id="1" /> {/* Still reuses cached data */}
      </OtherComponent>
    </>
  );
}

```

### Advanced Caching Patterns

```

// Combining multiple caching strategies
async function getDashboardData() {
  const [
    products = await fetch("/api/products", { cache: "force-cache" }),
    recentOrders = await fetch("/api/orders/recent", { cache: "no-store" }),
    analytics = await fetch("/api/analytics", { next: { revalidate: 300 } }),
  ] = await Promise.all([getProducts(), getRecentOrders(), getAnalytics()]);

  return {
    products: await products.json(),
    recentOrders: await recentOrders.json(),
    analytics: await analytics.json(),
  };
}
```

This caching system in Next.js 15 provides developers with fine-grained control over data freshness while maintaining optimal performance through automatic request deduplication and efficient cache invalidation strategies.

# What's New in Next.js 15

Next.js 15 introduces several significant improvements that enhance both development experience and application performance.

### React 19 Support

The framework now fully supports React 19 RC, bringing access to new React features and improved hydration error handling. Hydration errors now display detailed source code information with actionable suggestions for resolution.

### Turbopack Stability

Turbopack has reached stability in development mode, delivering impressive performance gains:

- 76.7% faster local server startup
- 96.3% faster code updates with Fast Refresh
- 45.8% faster initial route compilation

### Enhanced Security

Server Actions received significant security improvements:

- Automatic elimination of unused Server Actions
- Non-deterministic, unguessable IDs for client-side references
- Improved runtime configuration inheritance

Development Experience

- New Static Route Indicator helps identify static routes during development
- TypeScript support for `next.config.ts`
- ESLint 9 support while maintaining compatibility with ESLint 8

### Self-hosting Improvements

- Better control over Cache-Control directives
- Configurable `expireTime` value in `next.config`
- Automatic sharp integration for image optimization

These updates collectively make Next.js 15 a more robust, secure, and developer-friendly framework.

# Final Thoughts

Next.js 15 marks a significant evolution in modern web development, introducing crucial improvements in routing, server-side rendering, and caching mechanisms. The framework maintains its position as a leading choice for building performant web applications while providing developers with more explicit control over their applications' behavior.

Whether you're building a new project or considering an upgrade, Next.js 15's enhanced features and improved developer experience make it a compelling choice for teams focused on building fast, reliable, and maintainable web applications.

The future of web development continues to evolve, and Next.js remains at the forefront of this evolution, balancing innovation with practical development needs.

Next.js 15: A Deep Dive into Modern Web Development

Anthropic recently introduced their “Computer Use” [tool](https://docs.anthropic.com/en/docs/build-with-claude/computer-use), enabling developers to integrate their AI models to execute tasks on a computer. As they describe it in their blog [post](https://www.anthropic.com/news/3-5-models-and-computer-use). 

> Developers can direct Claude to use computers the way people do—by looking at a screen, moving a cursor, clicking buttons, and typing text.

Sounds cool, interesting, and a little scary. So we decided to kick the tires and see if this tool could help automate QA in any of our projects.

There’s three basic types of QA we do:

1. Exploratory, aka poke around and see if anything looks broken 
2. Verifying in-sprint development work
3. Building up a suite of regression tests

### TL;DR

The technology is very cool, but it's not ready for any production-level QA tasks. It is slow and often hits the rate limit of 40,000 tokens / minute very early in the testing flow.

### How we tested

To test this we ran a bunch of  experiments using Anthropics[ reference implementation](https://github.com/anthropics/anthropic-quickstarts/tree/main/computer-use-demo) in a Docker environment. The experiments were run on our pourwall.com marketing site and web app, as well as a few other projects. 

The reference implementation was easy to get running. After starting the tool in Docker, we  just needed to get an Anthropic API key and add it to the admin panel.

Each experiment was either an exploratory, in-sprint validation, or regression test. We added a prompt similar to what we’d ask a QA engineer to do, and just let it rip.  A few examples are found below.

### Experiment 1:  Exploratory website testing

**Prompt:** Navigate to pourwall.com, ensure that the page loads correctly and all the elements look OK. Record the results in a text file named pourwall_observations.txt. Then navigate to the FAQ and Plan pages and repeat for each. 

**Results:** [View here](https://drive.google.com/file/d/1vcIKw-3DIHWDtkHlZdsG87CoCAn_AgHG/view?usp=drive_link)

Succeeded in navigating to each page and validating that it was working. Ended up making lots of unhelpful recommendations and observations (more specific prompting would likely help)

**Notes:** Took roughly 2 minutes (video edited for brevity) and cost $0.15 (~150,000 input tokens and 2,000 output tokens)

### Experiment 2: Exploratory web app testing sign up

**Prompt:** Navigate to [https://pourwall.com/](https://dev.pourwall.com/) create a user account noting down the username and password in a text file. Login to the site, and perform exploratory analysis of the app. Make observations on bugs and potential fixes. 

**Results:** [View here](https://drive.google.com/file/d/19eCkk0yNaS68RoYFd2JAF9-ttuJB_d0c/view?usp=drive_link) 

The experiment failed due to token rate limiting. Used 40,000 tokens in a minute and didn’t even get signed up. Before hitting the rate limit, the tool struggled to complete the sign-up flow.

### Experiment 3: Exploratory web app testing login and use dashboard

**Prompt:** login to https://pourwall.com with credentials: username: redacted and password: redacted

Create a file called `pourwall_exploratory_observations.txt`

Perform exploratory testing on the app noting any bugs or odd behavior in the file. 

**Results:** Failed due to token limit

### Findings:

The tool does **not appear to be ready** for practical QA work, we won’t know more until we can get a higher token rate limit. At the moment we’ll be sticking with our current mix of human QA (for Exploratory and verifying in-sprint work) + scripted automated tests (for the suite of regression tests).

- Many experiments failed due to **token rate limiting early in the testing steps**. I requested a rate limit increase and will try again if they grant it.

- **Best potential use case:** Exploratory testing is probably the most likely use case. The tools' slowness, cost, and unpredictability don’t bode well for validating in-sprint work or suite or regression testing.

- The tool is cool, but still **slow and goofy**. [Watch it struggle to signup for Pourwall here.](https://drive.google.com/file/d/19eCkk0yNaS68RoYFd2JAF9-ttuJB_d0c/view?usp=drive_link)

- **High Costs:** Even a relatively simple task in Experiment 1 cost $0.15 for just one page of observations (not expensive compared to human QA, but will add up and be costly vs automated suite of tests)

### Conclusion

We’ll definitely re-do these experiments if our rate limit gets increased and will update the post if this happens. This technology is going to improve and we’re excited to come back and try again when it does.


Experimenting with Anthropic's Computer Use for QA

In this multi-part series, we’ll explore some of the latest and most impactful proposals from the JavaScript standards body. Each post will focus on a specific proposal, providing an in-depth look at its purpose, syntax, use cases, and potential impact on the JavaScript ecosystem. Currently, these proposals are in the TC39 staging process, which guides potential features from concept to adoption. Join us as we explore these exciting developments and examine how they could shape the future of JavaScript.

- **Array.zip and Array.zipKeyed:** Streamlining the process of combining arrays.
- **Atomics.pause:** Improving efficiency in multi-threaded environments.
- **Error.isError:** Providing a reliable method for error identification.
- **Iterator Chunking:** Managing large or infinite data streams efficiently.

### Array.zip and Array.zipKeyed

**Proposal:** [https://github.com/tc39/proposal-array-zip](https://github.com/tc39/proposal-array-zip)

**Stage:** 1

Introduced a proposal to add `Array.zip` and `Array.zipKeyed` methods. These methods aim to simplify the synchronization and combination of multiple arrays or iterables, enhancing code readability and performance.

This is closely related to the [Joint Iteration proposal](https://github.com/tc39/proposal-joint-iteration), effectively bringing `Iterator.zip` and `Iterator.zipKeyed` functionalities to arrays. By understanding this proposal, developers can anticipate and leverage upcoming features that make array manipulation more intuitive.

### Problem Statement

Despite the existence of iterators in JavaScript for over a decade, arrays remain the go-to data structure for most developers, especially for small and finite lists. Arrays are simple to work with, performant, and widely supported across the ecosystem. However, combining multiple arrays element-wise is not as straightforward as it could be.

Currently, merging arrays requires methods like Array.prototype.map, for loops, or external libraries such as Lodash. These approaches can be verbose, less ergonomic, and may introduce performance overhead.

For example, to merge two arrays element-wise:

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/2YBHYwoaSj6cYUWdSWBi)

While functional, this code isn't as clean or expressive as developers might prefer. There's a clear gap in JavaScript's array handling capabilities when it comes to synchronizing iteration over multiple arrays.

The `Array.zip` and `Array.zipKeyed` methods are based on the Joint Iteration proposal. The [Joint Iteration proposal](https://github.com/tc39/proposal-joint-iteration) introduces `Iterator.zip` and `Iterator.zipKeyed` to synchronize the iteration of multiple iterables.

By bringing these functionalities to arrays, the Array.zip proposal acknowledges the prevalent use of arrays over iterators in everyday JavaScript development. It offers the same powerful iteration synchronization but in a form that's more familiar and ergonomic for array users.

### Syntax and Examples

**Array.zip**

The `Array.zip` method creates a new array by synchronously iterating over multiple input arrays (or iterables) and combining their elements into tuples.

**Syntax**

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/CfC4oDdrRQO5QeGZS9Vh)

**Parameters**

- `iterables`: The arrays or iterables to zip together.

**Example**

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/LqJmdAwCQMSpnA0h3ied)

**Array.zipKeyed**

The `Array.zipKeyed` method synchronously iterates over multiple arrays (or iterables) and combines their elements into objects using a provided array of keys.

**Syntax**

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/tR8ap1ObS8WF0t7UvVXd)

**Parameters**

- `keys`: An array of strings representing the keys for the resulting objects.
- `iterables`: The arrays or iterables whose elements will become the values in the resulting objects.

**Example**

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/KVOii6aVQO2mEBqZ0RNP)

### Use Cases

**Data Merging:** When receiving separate arrays from different API endpoints that need to be combined into a single data structure.

**CSV Parsing:** After splitting CSV data into arrays, you can zip headers and values to create objects.

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/w8lu7y8ATGGmdd63OaGF)

**Form Handling:** Combining arrays of form field names and values into objects for processing or submission.

**Parallel Iteration:** Simplifying the iteration over multiple arrays where elements at the same index are related.

### Potential Pitfalls

**Uneven Array Lengths**

If the input arrays are of different lengths, `Array.zip` will only combine elements up to the length of the shortest array. The extra elements in longer arrays will be ignored.

**Example**

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/5tN7IUmtTwuoCkJND6FQ)

**Solution:** Ensure that all arrays are of the same length or handle the length discrepancies before zipping.

**Key Mismatch in Array.zipKeyed**

If the number of keys doesn't match the number of arrays provided, the method may not work as expected.

**Example**

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/kjtgT64ESE2aVW4MzLxD)

**Solution:** Ensure the number of keys matches the number of arrays to prevent missing or undefined properties.

**Not Yet Standardized**

`Array.zip` and `Array.zipKeyed` are still proposals and not part of the official ECMAScript standard. They may not be available in all JavaScript environments.

### Conclusion

The `Array.zip` and `Array.zipKeyed` proposal offers a more ergonomic and efficient way to synchronize and combine multiple arrays or iterables in JavaScript. By aligning with the Joint Iteration proposal, it brings powerful iteration capabilities to arrays, which are more commonly used than iterators in day-to-day coding.

Embracing these features can lead to cleaner codebases and more efficient data manipulation. Whether you're merging datasets, mapping keys to values, or performing parallel iterations, `Array.zip` and `Array.zipKeyed` offer intuitive solutions that enhance productivity.

Stay tuned for the next part of our series, where we'll explore how `Atomics.pause` can boost multithreaded performance in JavaScript.


Upcoming JavaScript Features: Simplifying Array Combinations with Array.zip and Array.zipKeyed

At **Beta Acid**, we've been experimenting with integrating Large Language Models (LLMs) into our workflows for over two years now. For our engineering team, GitHub Copilot has been a productivity enhancer, but we've recently started noticing some persistent flaws:

- Partial code completions
- Seemingly worsening suggestions
- Inconsistent use of context from the codebase
- Too much copy/pasting into chat windows or other LLMs


A few months ago we began exploring new tools that better integrate into our development workflow. One tool that stood out is Cursor, a fork of VS Code that integrates LLM features throughout the IDE. What sets [Cursor](https://www.cursor.com/) apart is its deep integration with context from your codebase, which dramatically improves code suggestions and completion accuracy.

## Harnessing the Power of Context

Cursor’s ability to leverage context is its strongest feature. It pulls context directly from your codebase by indexing files and directories, which allows it to understand your project’s structure and dependencies. In addition to this automatic indexing, you can give Cursor additional context by using specific references in chat, such as pointing to files, code snippets, or external documentation. 

Here are the main types of context that Cursor uses to improve your workflow:

- **Codebase Indexing:** Cursor indexes your entire codebase, providing it with some understanding of the project’s structure, functions, and classes. This allows the LLM to suggest code that is consistent with your project’s existing patterns and architecture.
- **`@files` and `@code`:** Use `@files` and `@code` to refer directly to specific files or blocks of code within your project. This ensures that Cursor can pull in relevant snippets or structural elements from your existing codebase.
- **`@docs`:** Calling `@docs` allows you to reference external documentation that Cursor can use to better understand your external dependencies. This is especially useful for libraries that change frequently and the LLM might have been trained on outdated material.
- **`@codebase`:** Use the `@codebase` symbol to reference your entire codebase contextually in chat. This is useful for tasks that require a broader understanding of your entire project, such as ensuring code consistency across different modules or suggesting improvements based on the overall architecture.

## Pro Tips for Using Cursor Effectively

Let’s walk through how Cursor can help us enhance our [FastAPI Reference App](https://github.com/betaacid/FastAPI-Reference-App), which calls the Star Wars API and demonstrates best practices in FastAPI development.

### 1. Use the Chat Feature for Narrow Tasks

Rather than relying on Cursor’s **Composer** feature (which generates an entire feature at once), use the **Chat** feature to tackle smaller, more manageable tasks. You can refine your instructions within the chat and apply them to your file step-by-step. Taking it slow—one small chunk of code at a time—will help you better understand the changes and avoid unwanted behavior.

Letting Cursor build entire features looks great in marketing demos, but is a recipe for trouble in professional development.

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/pFSJfZpyQJWYCBSCmvIe)

### 2. Choose the Right Model for the Task

For day-to-day code generation and chat interactions, we recommend using **Anthropic’s Claude 3.5 Sonnet.** It provides the best balance between quality, cost, and speed for routine development. For more in-depth architectural planning, enable usage-based pricing to utilize Open AI’s o1 model. It costs $0.40 per request but provides great results. 

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/J1VwbCfrQSW4Hv61frxL)

### 3. Add Custom Documentation for Better Context

If the documentation you need isn’t available by default in Cursor you can add it manually and Cursor will scrape and index it for future use. For example, Cursor doesn’t include [MJML docs](https://documentation.mjml.io/) out of the box, but you can have Cursor index the docs to improve code suggestions. Just press `CMD+L`, type `@Docs`, select "Add new doc", and provide the URL for the documentation. Cursor will scrape and index the content, allowing you to reference it in chat using `@docs`. 

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/g81beZkrREOW56Ow5Y8k)

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/uCS8QdrlTiiHs2UOpUZl)

### 4. Add a `.cursorignore` File to Skip Unnecessary Files

You can use a `.cursorignore` file to tell Cursor to ignore specific files or directories when using the `@codebase` command in chat. This prevents Cursor from pulling in irrelevant parts of your project, helping it focus on the files that matter most and providing cleaner, more relevant suggestions.

### 5. Define Team Preferences with `.cursorrules`

Use a `.cursorrules` file to standardize how Cursor interacts with your project. This file provides context for each request you make, ensuring consistent code styles and preferences across your team. Sharing these rules helps maintain a cohesive workflow. For inspiration, check out [this curated list](https://github.com/PatrickJS/awesome-cursorrules) of `.cursorrules` files that can be adapted to your team's needs.

### 6. Provide Clear and Detailed Instructions

When asking Cursor to generate code, provide as much relevant context as possible. For instance, in our reference app we’ve already build a characters endpoint, and now we want to add a new route to manage starships:

1. Create a new file for the route.
2. Press CMD+L to open the chat.
3. Describe your task: 
*"Create a new `/starships` route that retrieves a list of starships added in the last 24 hours. Use the existing `CharacterService` and follow the patterns from the `/characters` endpoint. Implement pagination to limit the results per page, include sorting by creation date, and ensure error handling for when no starships are found. Add validation to check that the created_at field is within the past 24 hours, and update the router to include the new route."*

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/Q2wXNMNXSYWVW6tObeYY)

Providing this level of detail will help you get 80% of the way to a working endpoint. Vague prompts like “Create a starships route” won’t follow your existing project structure.

### 7. Use Notepads for Common Tasks

If you’re frequently asking Cursor to perform similar tasks, the Notepads feature can streamline your workflow. For example, create a Notepad for “AddingNewRoute” in your Star Wars API. This could include common patterns like router structures, service layers, and database calls.

To set up a Notepad, go to Composer > Control Panel > Add Notepad, and store relevant code snippets and file references. When you’re ready, simply reference it in chat using @notepad for consistency across tasks.

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/xBujFRPRTaCpu2L1U0lI)

### 8. Don’t Blindly Trust Results

After receiving multiple accurate results from Cursor, it can be tempting to start accepting suggestions without a thorough review. This is a mistake. LLMs can still introduce bugs and hallucinate, particularly when refactoring or suggesting code changes.

- **Bugs:** Even after several successful interactions, review everything carefully. We’ve encountered cases where Cursor changed a file path in a refactor without realizing the template it altered didn’t match.
- **Non-Optimal Solutions:** Cursor may suggest viable but inefficient solutions for your platform. Rely on your expertise to adjust these suggestions to follow best practices.

## Conclusion

At **Beta Acid**, we’re always looking for ways to boost productivity, and while Cursor is a great tool, it’s not magic—it’s simply a smarter interface for working with LLMs. It helps streamline workflows, but ultimately, it's you, the developer, driving the results. Use Cursor’s features to your advantage, but stay grounded—thoughtful engineering always comes first.

Cursor dethrones Copilot

At Beta Acid, we believe in building software like a startup—fast, efficient, and adaptable. Today, we're excited to open-source [a reference API app](https://github.com/betaacid/FastAPI-Reference-App) that showcases our clean, modular FastAPI architecture designed for both agility and robustness. FastAPI is a popular Python web framework for building APIs quickly.

## Building Fast Without Compromise

Speed is a key factor in software development, offering a distinct advantage by bringing products to market faster. But it’s just as important to balance this with strong code quality and maintainability. When collaborating with larger, more established businesses, the emphasis shifts toward rigorous testing and ensuring scalability to support an expanding user base.
Our FastAPI reference app strikes a balance between rapid development and long-term reliability. It enables developers to create applications quickly while maintaining a clean codebase that's easy to test, scale, and modify with confidence.

## What's Inside?

Our reference app is a Star Wars character API that allows users to enter a character name, fetch details from the [SWAPI API](https://swapi.dev/), and store them in a database. While the app's functionality is straightforward, it embodies architectural best practices suitable for projects of any size.

### Key Features:

- **Clean, Modular Structure:** Each component of the application is designed with a clear separation of concerns, making the codebase intuitive and maintainable.
- **Robust Testing Strategy:** High test coverage is achieved through a well-defined testing pyramid, ensuring each module works correctly both in isolation and together.
- **Scalability:** The architecture is designed to be scalable, allowing for easy expansion as project requirements grow.
- **Ease of Maintenance:** With clear naming conventions and modular components, developers can confidently make changes without fearing unintended side effects.

## The Architecture Breakdown

### Main File

The `main.py` file serves as the application's entry point. It initializes the FastAPI app and includes the necessary routers.

```
from fastapi import FastAPI
from app.routers import characters_router

app = FastAPI()
app.include_router(characters_router)
```

### Routers

Routers define the API endpoints and manage request validation, keeping the logic clean by delegating core business operations to the service layer. This helps ensure the routers stay focused on routing and error handling, without becoming overloaded with business logic.

```
@characters_router.post("/", response_model=StarWarsCharacterRead)
async def create_character(input_character: StarWarsCharacterCreate, db: Session = Depends(get_db_session)) -> StarWarsCharacterRead:
    return add_new_character(input_character, db)
```

### Service Layer

The service layer acts as the core coordinator of business logic. It processes input data, applies business rules, and orchestrates the flow between different components. External API calls and database interactions are delegated to the clients, keeping the service layer decoupled from external dependencies and enhancing testability.

```
def add_new_character(input_character: StarWarsCharacterCreate, db: Session) -> StarWarsCharacterRead:
    swapi_json = get_character_from_swapi(input_character.name)
    swapi_character = transform_swapi_character_json_to_pydantic(swapi_json)
    formatted_name = format_star_wars_name(swapi_character.name)
    swapi_character.name = formatted_name
    return insert_new_character(db, swapi_character)
```

### Clients

Clients handle interactions with external services and databases. They encapsulate the logic for making API calls or database operations, providing a clean interface for the service layer to use. This isolation enhances modularity and makes the codebase easier to test and maintain.

For example, the client for interacting with the SWAPI API:

```
def get_character_from_swapi(name: str) -> dict:
    response = requests.get(f"https://swapi.dev/api/people/?search={name}")
    response.raise_for_status()
    return response.json()
```

Similarly, database interactions are managed by database clients or repositories:

```
def insert_new_character(db: Session, character: StarWarsCharacter) -> StarWarsCharacterRead:
    db_character = StarWarsCharacterModel.from_pydantic(character)
    db.add(db_character)
    db.commit()
    db.refresh(db_character)
    return db_character.to_pydantic()
```

### Domain Logic

Domain logic contains the core business rules and calculations, ensuring that the application's fundamental operations are consistent and reliable. By centralizing business logic, we make it reusable and easier to manage.

```
def convert_consumables_to_days(consumables: str) -> int:
```

### Utils

Utility functions provide reusable, stateless operations that support various parts of the application. These are common functions that don't fit neatly into other categories but are essential for code reusability.

```
def format_star_wars_name(name: str) -> str:
```

## Benefits of This Architecture

### Confidence in Code Changes

With a robust testing strategy and clear separation of concerns, developers can modify and extend the codebase with confidence. Changes are validated at multiple levels, reducing the risk of introducing bugs.

### Scalability

The modular design and testing strategy make scaling the application straightforward. As new features are added, existing tests provide assurance that current functionality remains unaffected.

### Maintainability

Clear naming conventions, a well-organized directory structure, and a comprehensive test suite make the codebase easy to navigate and maintain. This reduces onboarding time for new developers and simplifies ongoing maintenance.

### Testability

By isolating external dependencies and focusing on testing public methods, the architecture enables comprehensive and efficient testing. This ensures each component functions correctly both individually and as part of the whole system.

### Flexibility

The architecture is adaptable to different project needs, whether it's rapid prototyping or building enterprise-level applications. The principles applied here can be extended or customized to fit various scenarios.

## Our Testing Pyramid Approach

A key aspect of our architecture is the emphasis on a testing pyramid approach. This strategy ensures a solid foundation of high-quality, reliable code, allowing us to move quickly without sacrificing stability.

### Unit Tests at the Base

At the base of the pyramid are **unit tests**, which provide the bulk of our test coverage. These tests are:

- **High coverage:** We write comprehensive unit tests for 80%+ of public methods, ensuring that each component works correctly in isolation.
- **Fast and Reliable:** By mocking external dependencies, unit tests run quickly and provide immediate feedback.
- **Focused:** Each test targets a small piece of functionality, making it easier to pinpoint issues.

### Integration Tests in the Middle

Above the unit tests are **integration tests**, which are fewer in number but cover critical pathways. For an API like this, our integration tests:

- **Hit the Actual API Endpoints:** We test the application by making real HTTP requests to our API, simulating client behavior.
- **Use Real External Services and Databases:** These tests interact with actual external services like the SWAPI API and use the database to verify that all components work together as expected.
- **Verify End-to-End Functionality:** They ensure that data flows correctly through the system, from the API request to the database and back.

In most real-world situations, we take integration testing a step further.

### Front-End Integration Testing

When our applications include a front end, we employ tools like [Playwright](https://playwright.dev/) for end-to-end testing. This allows us to:

- **Test the Front End and Back End Together:** By automating browser interactions, we verify that the user interface works seamlessly with the live API.
- **Uncover Integration Issues:** These tests can catch problems that unit or back-end integration tests might miss, such as mismatches between API responses and front-end expectations.

### Our QA Team stays at the top

By investing heavily in unit tests and strategic integration tests, we free up our QA team to focus on:

- **Quick feedback on in-sprint work:** QA tests new features as they are finished by the development team. 
- **Writing New Integration Tests:** After the initial testing, they can add to the front end test scripts
- **Exploratory Testing:** With confidence in the automated test coverage, QA can spend the rest of their time exploring edge cases and performance issues.

This approach not only improves code quality but also accelerates the development cycle, as issues are caught early and can be addressed promptly.

## Conclusion

At Beta Acid, we're committed to combining speed with quality. Our FastAPI reference architecture and testing pyramid approach embody this commitment, providing a foundation that is both agile and robust. By focusing on high unit test coverage, strategic integration tests, and comprehensive front-end testing with tools like Playwright, we ensure that our codebase is reliable and our QA team is empowered to enhance the product further.
We invite you to explore the codebase, utilize it in your projects, and contribute to its growth. Together, we can build software that's both fast and dependable.

*Explore more of our projects and insights on [our blog](https://betaacid.co/blog). If you have questions or are interested in collaborating, feel free to reach out—we'd love to hear from you.*

Introducing Our Clean and Modular FastAPI Reference Architecture

Sergio Rodriguez

In the fast-paced world of product design, efficiency, consistency, and scalability are paramount. At Beta Acid, we harness the power of design systems to streamline our processes, deliver high-quality products, and maintain a strong brand identity for our clients. This post will explore the major advantages of using design systems in product design, illustrated by a recent client case study.

## What is a Design System?

A design system is a collection of reusable components, guided by clear standards, that can be assembled to build any number of applications. It serves as a single source of truth for designers and developers, ensuring consistency across different products and platforms.

Design systems are highly adaptable tools that can be used to create a wide range of products and materials. They can be applied to everything from digital interfaces like websites and mobile apps to marketing assets, internal documentation, and even cutting-edge technologies such as augmented and virtual reality.

### 1. Consistency Across Products

Design systems ensure uniformity in the look and feel of all products. This consistency not only strengthens the brand identity but also enhances the user experience. Reusable components maintain design integrity across various applications, making every interaction familiar and intuitive.

### 2. Efficiency and Speed

By using predefined components and patterns, our team at Beta Acid can significantly reduce the time spent on creating and coding elements from scratch. This streamlined process allows us to deliver projects faster without compromising quality.

Shopify reduced their [development cost by 50%](https://www.efigence.com/ux-ui/design-system-business-impact/) by using [Polaris](https://polaris.shopify.com/), their own design system, to scale their features and make their application consistent.

The data science team at Figma set out to find out just how much time (and money) could be saved when teams leverage a design system. As a result, [designers completed their task 34%](https://www.figma.com/blog/measuring-the-value-of-design-systems/) faster with a DS implemented.

### 3. Scalability

A well-structured design system can scale with the growth of the product. Changes made in the design system can be propagated across all products that use it, ensuring consistent updates and easy adaptability for new features and platforms.

### 4. Improved Collaboration

Design systems provide a common language for designers, developers, and other stakeholders. At Beta Acid, this common framework facilitates better communication and collaboration, ensuring that all team members are aligned with the project’s vision and goals.

### 5. Enhanced User Experience

Consistency in design patterns helps users understand and predict interactions, leading to a more intuitive user experience. Incorporating accessibility standards within the design system ensures that our products are inclusive and usable by a wider audience and compliant with global guidelines.

### 6. Cost-Effectiveness

Reducing the need for repetitive work allows our team to allocate resources more effectively. By focusing on innovation and higher-level design challenges, we can deliver more value to our clients. Additionally, standardized components simplify long-term maintenance, reducing overall costs.

### 7. Quality Assurance

With a design system, quality control becomes more straightforward. All components are built and tested to meet predefined standards, ensuring a higher level of quality and performance in the final product.

### 8. Brand Integrity

A strong brand identity is crucial for any business. Design systems ensure that all visual and interaction elements align with the brand strategy, reinforcing the brand’s visual identity and making it more recognizable and trustworthy.

## The Anatomy of a Design System

Recently, Beta Acid completed a project for real estate client Maymont for their rollout of Build To Rent (BTR) offerings. The challenge was to create a series of community websites that were consistent, efficient, and scalable.

### 1. Foundations

A design system's foundation establishes its visual identity. This includes defining the core visual elements such as color palettes, typography, and iconography. It also encompasses the product's overall tone and voice, as well as essential guidelines for accessibility and brand consistency. Our work on the Maymont Build To Rent (BTR) project integrated these foundational elements directly into the component library for easy reference and application.

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/t2KkQbG8QIiTmTttw70c)

### 2. Components

A design system's component library is a centralized repository of pre-designed, reusable UI elements. It serves as a comprehensive resource for both designers and developers to understand and implement specific UI components efficiently. Building a robust component library requires substantial time and effort. For the Maymont BTR project, we developed a component library encompassing the following elements:

- **Component Naming**: Clear and distinct names for each UI component to prevent misunderstandings between design and development teams.
- **Component Descriptions:** Detailed explanations of each component's purpose, usage, and best practices, including do's and don'ts for guidance.
- **Component Customization:** Customizable settings, such as color, size, and typography, to adapt components to various design needs.
- **Component States:** Defined default appearances and variations for different interactive states (hover, active, disabled, etc.).
- **Front-End Integration:** Compatibility with specific front-end frameworks to streamline implementation and reduce troubleshooting.   

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/qx7CdBVsRPqRPwcb8IJh)

### 3. Usability Patterns

Pattern libraries showcase pre-defined combinations of UI elements or layout structures. While often perceived as less detailed than component libraries, they can be equally comprehensive depending on project requirements. The Maymont BTR project includes pattern libraries for content structures, layouts, and theming. Similar to components, these patterns are designed for reusability and customization.

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/ROo0AhVQtWtTZo5E6Mvk)

### 4. Implementation

The decision to invest in a custom design system depends on an organization's specific needs. While pre-existing, open-source design systems offer cost-effective solutions, extensive customization often outweighs these initial savings. In such cases, building a tailored design system becomes more advantageous. For Maymont BTR, we determined that a custom design system was essential to effectively design and theme multiple communities based on a shared foundation.

 ![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/CXTv7eyVRvukCDL8qnA4)

## Conclusion

At Beta Acid, design systems are an integral part of our product design process. They offer numerous advantages, from ensuring consistency and efficiency to enhancing user experience and maintaining brand integrity. Our recent work with Maymont BTR is a testament to the power of design systems in delivering high-quality, scalable, and cost-effective solutions.

If you’re looking to streamline your product design process and achieve consistent, high-quality results, contact Beta Acid today. Let us show you how our design systems can transform your projects.

A Blueprint for Efficient Design

Design Systems

Araminta Atherton

In today's ever-evolving business landscape, companies face continual challenges driven by rapid transformations, including technological advancements, shifting consumer preferences, and global market dynamics. Amidst this dynamic backdrop, design often remains undervalued or overlooked within organizations. However, for business leaders, it's essential to recognize that design transcends mere aesthetics; it is a fundamental component of any product or service.

At Beta Acid, we believe that design is not just about making things look good—it’s about making things work better. 

> In the words of Steve Jobs, "Design is not just what it looks like and feels like. Design is how it works."
> 

Here’s why design should be at the core of your business strategy.

### **The Power of First Impressions**

First impressions matter. In the digital age, your product's interface is often the first touchpoint a customer has with your brand. Whether it’s a sleek and intuitive mobile app or a sophisticated web platform, good design makes a lasting impression. According to studies, users form an opinion about a product within 50 milliseconds of encountering it. A well-designed interface can capture attention, build trust, and invite engagement, whereas poor design can lead to immediate disengagement. Therefore, investing in quality design can significantly enhance user acquisition and retention rates.

### **Enhancing User Experience (UX)**

User experience is the cornerstone of customer satisfaction. When design is seamlessly integrated into the development process, it ensures that every interaction a user has with the product is intuitive and enjoyable. This involves understanding the user’s journey and designing interfaces that facilitate ease of use, accessibility, and efficiency. A product that solves problems elegantly and effortlessly keeps users coming back, drives word-of-mouth referrals, and reduces churn rates.

Robert Pressman, in his book *Software Engineering: A Practitioner’s Approach*, states early [business justification](http://nlp.chonbuk.ac.kr/SE/pressman07book.pdf) for UX: “For every dollar spent to resolve a problem during product design, $10 would be spent on the same problem during development, and multiply to $100 or more if the problem had to be solved after the product’s release.” This is to say that every dollar invested in UX returns 10 to 100x, underscoring the importance of correcting problems from the start to be most efficient and cost effective.

> "Obsess over customers. When given the choice of obsessing over competitors or customers, always obsess over customers. Good design thinking puts the customer at the center of everything you do." - Jeff Bezos, Founder of Amazon
> 

### **Differentiating from the Competition**

In saturated markets, differentiation is key. Design can be a powerful differentiator that sets your product apart from competitors. It's not just about being different for the sake of it but providing a unique value proposition communicated through thoughtful design. For example, in the crowded space of social networking apps, a platform that prioritizes user-centric design—offering features that users didn’t even know they needed but soon can’t live without—will stand out. By leveraging design as a strategic tool, you can carve out a niche and create a loyal customer base.

### **Driving Innovation**

Design is a catalyst for innovation. It encourages a culture of creativity and problem-solving within an organization. By fostering collaboration between business leaders, designers, and engineers, companies can explore new ideas, test hypotheses, and iterate rapidly. This iterative process of design thinking leads to innovative solutions that not only meet but exceed user expectations. Beta Acid’s collaborative approach ensures that clients, design, and engineering work hand-in-hand to deliver leading-edge products.

### **Aligning with Business Goals**

Strategic design aligns with your broader business goals. It involves understanding your market, your competition, and your customers, and using these insights to inform design decisions. For instance, a real estate app designed to streamline property management must address specific pain points experienced by landlords and tenants. By aligning design with these goals, you create a product that delivers tangible business value. This alignment ensures that design is not an afterthought but a driving force behind your business strategy.

### **Boosting ROI Through Professional Design**

Investing in professional design can significantly boost your return on investment (ROI). A well-designed product can command higher prices, reduce customer support costs, and increase sales and market share. Here are some key areas where professional design impacts ROI:

1. **Customer Engagement**
Professional design enhances customer engagement by creating intuitive and enjoyable user experiences. For example, a study by [Forrester Research](https://www.forrester.com/report/The-Six-Steps-For-Justifying-Better-UX/RES117708) found that a well-designed user interface could increase conversion rates by up to 200%, while a better UX design could yield conversion rates up to 400%. This illustrates the significant impact that design can have on how users interact with a product.

2. **Brand Loyalty**
Design plays a crucial role in building brand loyalty. Consistent and appealing design across all touchpoints fosters a strong brand identity, making customers more likely to develop an emotional connection with your brand. Apple is a prime example of this. Their meticulous attention to design across all products and services has cultivated a loyal customer base that is willing to pay premium prices.

3. **Sales Growth**
The impact of professional design on sales growth is profound. For instance, research by [McKinsey & Company](https://www.mckinsey.com/capabilities/mckinsey-design/our-insights/the-business-value-of-design) highlights that companies investing in design see revenues increase by 32% and shareholder returns by 56% over a five-year period compared to those that don’t. This underscores how strategic design investment directly correlates with financial performance.

### **Real-World Examples**

Let's take a look at some real-world examples to further illustrate the ROI of design in business:

- **Airbnb**: By redesigning its website and mobile app to focus on user experience, Airbnb was able to significantly improve its booking process, leading to increased user engagement and higher booking rates. This design overhaul contributed to Airbnb's growth from a small startup to a global hospitality giant.

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/DbB8XyM3TwmdX4q4tw14)

- **Spotify:** Spotify revolutionized the music streaming industry with its user-friendly design and personalized user experience. The platform's sleek, intuitive interface makes it easy for users to discover and enjoy music. Spotify's design team focuses on creating seamless experiences across devices, ensuring that users can easily access their music wherever they are. The platform's success is also due to its sophisticated algorithms, which provide personalized recommendations, making Spotify a preferred choice for millions of users worldwide.

- **Squarespace:** Squarespace has distinguished itself in the crowded website building market by focusing on high-quality design. The platform offers beautifully designed templates that allow users to create professional-looking websites with ease. Squarespace's emphasis on aesthetics and user experience has made it a popular choice for creatives, entrepreneurs, and small businesses. The company’s commitment to design extends to its branding and advertising, which consistently showcase the platform’s elegant and modern design capabilities.

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/45ZSF1bQTPKjRIoR67ys)

- **Calm:** Calm has emerged as a leading wellness app by prioritizing a serene and engaging design that enhances the user experience. The app’s interface is designed to be calming and intuitive, offering users easy access to a variety of mindfulness and meditation resources. Beautiful visuals, soothing sounds, and seamless navigation contribute to the app's tranquil atmosphere, encouraging regular use. Features like personalized meditation plans, sleep stories, and guided sessions are thoughtfully integrated to meet diverse user needs. Calm’s dedication to superior product design has helped it build a loyal user base and establish itself as a trusted brand in the mental health and wellness space.
    
![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/tW7DihNRNKmTBcWvYnBF)

- **Slack**: The workplace communication tool Slack invested heavily in its design to create an intuitive and enjoyable user experience. This focus on design helped Slack differentiate itself from traditional communication tools, resulting in rapid user adoption and growth. Slack's user-friendly design has been a key factor in its success and widespread popularity.

### **Fostering Emotional Connections**

Great design fosters emotional connections with users. It’s about creating an experience that resonates on a deeper level, evoking positive emotions and building brand loyalty. In industries like hospitality and e-commerce, where customer experience is paramount, design can be the differentiating factor that turns a one-time user into a lifelong advocate. By prioritizing design, you not only meet functional needs but also create memorable experiences that leave a lasting impact.

### **Conclusion**

Design is an indispensable element of modern business strategy. It’s not just a superficial layer added to a product; it’s an integral part of the development process that can drive significant business value. By investing in thoughtful, user-centric design, businesses can create products that are not only beautiful but also highly functional and impactful.

At Beta Acid, we’re committed to helping you leverage design to get ahead of your competition. Whether you’re building a SaaS application, an e-commerce platform, or any other digital product, our team of expert designers and engineers work together to deliver exceptional results. Let us help you build impactful and intuitive digital products that not only meet but exceed your business goals. Hit us up to learn how we can elevate your business through strategic design.

Gaining a Competitive Edge

Design as a Business Strategy:

Ryan Vanderpol

Like Alec Baldwin’s famous mantra in Glengarry Glen Ross, we have a phrase we live by: Always Be Delivering. 

Everything that everyone does at Beta Acid should be in support of delivering a tangible product or some other real world value to the teams we work with. It’s what separates our style of working from other design and engineering teams. 

You may have heard me talk about this mantra using a different phrase: Get Shit Done.

Honestly, this is our business model. Companies hire us to deliver value and they expect to get that value quickly and efficiently. 

To us, "Always Be Delivering" means that every member of our team is constantly focused on pushing projects forward, from the initial design phase to the final launch. It’s about maintaining momentum, ensuring progress, and never losing sight of the ultimate goal: delivering value.

### Clarity in Ambiguity

One of the ways we try to deliver value early in our engagements is by creating clarity in ambiguous situations. We thrive in uncertain environments. At least half of the projects that drop into my inbox are… what comes before “half-baked”??

It’s our responsibility to structure conversations, ask the right questions, and collect the data we need to figure out how the pieces fit together. This allows us to envision the final state of a project, define the scope, and document a roadmap for our team to deliver it.

### Iterative Processes

Another way we continually deliver is by using an iterative design and development process. Practically speaking, this means that our creative and engineering teams don’t go dark for long periods of time. We quickly craft our ideas and work with our client’s team to review them, collecting immediate feedback that we can use to revise and improve our work.

When we say we build software using the lean, agile processes that startups use this is a key part of it. Most of the projects we work on are net new products and it’s critically important to our clients that we get those products into market quickly to collect feedback so we can figure out what works and what doesn’t work. We don’t strive for perfection; we strive for an educated balance of quality and speed to get you to market fast.

### Sense of Urgency

Most of the teams we work with are anxious to get their products built and launched and we are required to match that energy. In many cases, every day that our work is not complete is lost revenue. Every extra hour we work is more cost to our clients. Our clients expect us to make the most of every minute we spend on their projects and that is exactly what we do.

This requires us to be thoughtful, but also focused. To be excellent at prioritization and be proficient decision makers. To feel ownership for every project we take on and act as if it were our own.

### Communication

It’s probably cliche to say “communication is key” but clear, consistent, and quick communication is incredibly important both with our clients and with our internal team. Communication with each other ensures that we are spending our full energy working on the right things, which quickens our pace of delivery. 

When we have questions or concerns, it is crucial that we communicate quickly to resolve these issues so that we can continue providing the fullest amount of value that we can. Communication also fosters a collaborative environment which allows everyone on the team to work together with higher efficiency.

### Big Hearts

We understand that the teams we work with are entrusting us to help them bring their dreams to life and we take that trust very seriously. It’s both central to our philosophy and the reason why we do what we do. Everyone on our team acts with urgency and a sense of responsibility because we love what we do and we want to see every product we work on and every team we work with be successful.

What can we deliver for you?

Like Alec Baldwin’s famous mantra in Glengarry Glen Ross, we have a phrase we live by: Always Be Delivering. 

Always Be Delivering

Gastón Barbero

When working on data science projects, measuring a model's performance is key to determine whether the resulting model is ready to make it to production. However, one key factor that is barely taken into account when carrying out said measurements is incorporating business metrics into the model training and fine tuning process. This approach not only aligns technical outputs with business objectives but also bridges the gap between data science teams and business stakeholders. It also ensures that machine learning initiatives are not just technically sound but also strategically significant, fostering a data-driven culture where insights are actionable and investments in AI and machine learning yield substantial returns.

This strategy is applicable in different fields, such as predicting the severity of an illness to correctly allocate medical resources for treatment or predicting demand on perishable goods to minimize waste and maximize gains. To demonstrate how we can make use of these business defined metrics we will be training a model on the [Credit Card Fraud Detection](https://www.kaggle.com/datasets/mlg-ulb/creditcardfraud) dataset which contains real, yet anonymized, credit card transactions. This is a typical binary classification problem with a highly imbalanced dataset, where our main goal is to predict if a transaction is fraudulent or not.

In a usual machine learning workflow we would start by defining the target metrics to optimize when training the model. Given the type of problem, we would use metrics like the [F1 score](https://en.wikipedia.org/wiki/F-score), precision, recall or [Matthew’s correlation coefficient](https://en.wikipedia.org/wiki/Phi_coefficient). However, more often than not, when sharing results with stakeholders most data scientists simply rely on indicating that if there is an uplift in X metric, that means we have a better model. But stakeholders speak KPIs and business metrics. For this problem, we will measure a model’s performance in terms of Net Gain, being defined as:

> Net Gain =Transaction Amount * Fee Percentage- Fraudulent Transaction Amounts

Assuming we are a credit card provider, we would charge a fixed rate on each transaction, let’s say 2%. On the other hand, for each fraudulent transaction its entire amount represents a loss to our business. We now need to determine how we plug it in our model training process. In order to do so, we first need to reason how our model would impact the business when in production:

- If our model predicts the transaction as non fraud and:
  - It isn’t *fraudulent* (a true negative): we’d earn that transaction’s fee
  - It is *fraudulent* (a false negative): we’d earn that transaction’s fee, but lose the entire transaction’s amount
- If our model predicts the transaction as fraud and:
  - It is *fraudulent* (a true positive): we’d lose that transaction’s fee (but we would have avoided the loss of the entire transaction’s amount)
  - It isn’t *fraudulent* (a false positive): we’d lose that transaction’s fee

By following the aforementioned rules, we can compute the Net Gain at a transaction level. We then sum up each transaction’s Net Gain to get the overall metric value. This is a special case of [cost-sensitive learning](https://machinelearningmastery.com/cost-sensitive-learning-for-imbalanced-classification/).

Time to implement this in code. Let’s start with the setup, a basic requirements.txt file:

```
pandas==2.2.2
scikit-learn==1.4.2
xgboost==2.0.3
```

For the sake of simplicity, we will skip all the exploratory data analysis and jump directly into training the model. First, let’s import all the required libs:

```
from typing import Iterable

import numpy as np
import pandas as pd
from numpy.typing import NDArray
from sklearn.model_selection import StratifiedKFold
from xgboost import XGBClassifier
```

Now, we’ll load the data and split it into the training and testing sets:

```
data = pd.read_csv("creditcard.csv")

splitter = StratifiedKFold(shuffle=True, random_state=42)
train_ix, test_ix = next(splitter.split(data, data["Class"]))

train = data.iloc[train_ix]
test = data.iloc[test_ix]

X_train = train.drop(columns=["Time", "Class"])
X_test = test.drop(columns=["Time", "Class"])
y_train = train["Class"].values

# We'll also keep the amount in the testing
# ground truth so we can compute the net gain
# later
y_test = test[["Amount", "Class"]].values
```

Next, let’s establish the baseline, which is the Net Gain of the business without the model:

```
FEE_RATE = .02

total_loss = test[test["Class"] == 1]["Amount"].sum()
total_earnings = test["Amount"].sum() * FEE_RATE
net_gain = total_earnings - total_loss

print(f"Total earnings: {total_earnings:>12.2f}")
print(f"Total fraud losses: {total_loss:.2f}")
print(f"Net gain: {net_gain:>18.2f}")

>>> Total earnings:      98730.76
>>> Total fraud losses: 15186.50
>>> Net gain:               83544.26
```

Our goal will then be to produce a Net Gain higher than $83,544.26 using our model. But before training the model, we need to define our function to compute the business metric:

```
def compute_net_gain(
    y_true: NDArray[tuple[float, int]],
    y_preds: NDArray[int],
) -> float:
    net_gain = 0
    for (amount, class_), y_pred in zip(y_true, y_preds):
        # True positive
        if y_pred == class_ == 1:
            net_gain -= amount * FEE_RATE

        # True negative
        elif y_pred == class_ == 0:
            net_gain += amount * FEE_RATE

        # False positive
        elif y_pred != class_ == 0:
            net_gain -= amount * FEE_RATE

        # False negative
        elif y_pred != class_ == 1:
            net_gain += amount * FEE_RATE - amount

    return net_gain
```

With that in place, let’s train an eXtreme Gradient Boosting algorithm on our training dataset, generate predictions for our testing dataset and finally plug the generated predictions as well as the testing ground truth into our previously defined function:

```
clf = XGBClassifier(random_state=42).fit(X_train, y_train)
y_pred = clf.predict(X_test)
compute_net_gain(y_test, y_pred)

>>> 94543.086
```

Should our model have been in production, the company’s Net Gain would have been $94,543.086, an outstanding **13.16%** lift from the $83,544.26 baseline.

Finally, we can do a threshold adjustment to maximize the gains and pick the best cutoff value:

```
y_probs = clf.predict_proba(X_test)[:, 1]
net_gains = []

thresholds = np.arange(.01, 1, .01)
for threshold in thresholds:
    y_pred = (y_probs >= threshold).astype(int)
    net_gain = compute_net_gain(y_test, y_pred)
    net_gains.append(net_gain)

max_gain = max(net_gains)
best_threshold = thresholds[net_gains.index(max_gain)]
print(f"Max net gain: {max_gain:.3f}\nat threshold: {best_threshold}")

>>> Max net gain: 94865.504
>>> at threshold: 0.03
```

By using a .03 threshold, we now achieve a Net Gain of $94,865.50, representing a **13.55%** lift from our baseline.

![Chart of thresholds and their impact on Net Gain](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/sDyo1xwVSE2QXV2fwgZP)

Communicating the 13.55% net gains lift is more impactful than detailing machine learning metrics like F1 or precision. This tangible financial improvement directly resonates with stakeholders, clearly demonstrating the model's value in enhancing profitability and operational effectiveness. This approach aligns with business goals, making the benefits of our data science solutions easily understandable and relevant to decision-makers.


Using business metrics to measure a model's performance is an often overlooked but essential method for fine tuning.

Measuring Data Science Models

Sentiment Analysis is certainly not a new concept, but with powerful modern AI tools it is becoming incredibly easy to do. Here we’ll go through what it is, common use cases for it, and how we can gather actionable data from natural language with just a few lines of code.

For those who are unfamiliar with the concept, Sentiment Analysis (sometimes called “opinion mining”) is the computational process of determining the emotional tone behind a series of words. This is used to gain an understanding of the attitudes, opinions, and emotions expressed within an online mention, be it in social media posts, reviews, forums, or other digital platforms. This emotional tone is typically categorized as positive, negative, or neutral.

GPT-4 is a Large Language Model (LLM) which, by default, returns unstructured natural language as its response. However, OpenAI recently released a new feature called [Function Calling](https://platform.openai.com/docs/guides/function-calling) which you can use to instruct the LLM to return data in a structured manner (eg. JSON) so that the code you build to call the LLM can, in turn, pass the result off to another function with consistent results.

Using the tools we’ll discuss here, you can build automated sentiment analysis into your product or company process. Automated sentiment analysis, especially in bulk, can be a huge help for companies to prioritize product feedback and identify what customers to respond to. 

## Common Use Cases for Sentiment Analysis
There are a lot of use cases for performing Sentiment Analysis, but in general it is used to either get a sense of the average sentiment or trending sentiment about a topic, or to quickly identify negative sentiment that needs to be addressed across a broad dataset.

Here are some more practical examples of how sentiment analysis is used by companies today.
##### Brand Monitoring:
Companies use sentiment analysis to monitor social media and other online platforms to understand public opinion about their brand, products, or services. This helps in managing brand reputation and responding quickly to negative sentiment trends.
##### Customer Feedback Analysis:
Businesses analyze customer reviews and feedback on their products or services to glean insights about customer satisfaction and identify areas for improvement.
##### Market Research:
Sentiment analysis is used in market research to understand consumer attitudes towards products, services, or concepts. This can inform product development, marketing strategies, and competitive positioning.
##### Political Campaigns and Public Opinion:
Politicians and public organizations use sentiment analysis to gauge public opinion on policies, campaigns, and social issues. This can influence policy-making and campaign strategies.
##### Financial Markets:
In the financial sector, sentiment analysis of news articles, reports, and social media can predict market trends and potential stock movements. Trading performed on this information is at your own risk!

## Build a Sentiment Analysis Profiler
Most of our previous articles about AI and Machine Learning tools have highlighted building in Python, which is probably the most common language used for these types of tasks, but many of the products we build use a Javascript stack, so we’ll do this one in Typescript this time.

We’ll be using OpenAI’s Function Calling, accessible via the `tools` parameter. As of today, Function Calling is only accessible to specific GPT models, so we’ll be using `gpt-4-1106-preview`. To use the GPT-4 model you’ll need an OpenAI account and API Key.

The code is actually fairly simple and looks as follows:

```
const determineSentiment = async (prompt: string): Promise<'positive' | 'negative' | 'neutral'> => {
 const chatResponse = await openai.chat.completions.create({
   model: "gpt-4-1106-preview",
   messages: [
     {
       role: "user",
       content: `Determine the sentiment of the following text: "${prompt}". Is it positive, negative, or neutral?`,
     },
   ],
   tools: [
     {
       type: "function",
       function: {
         name: "extractSentiment",
         description: "Extract the sentiment from the user prompt.",
         parameters: {
           type: "object",
           properties: {
             sentiment: {
               type: "string",
               enum: ["positive", "negative", "neutral"],
               description: "The sentiment of the text.",
             },
           },
         },
       },
     },
   ],
 });

 const toolCalls = chatResponse.choices[0].message?.tool_calls;
 if (toolCalls && toolCalls?.length > 0) {
   return JSON.parse(toolCalls[0].function?.arguments).sentiment;
 }
 throw new Error("Failed to determine sentiment");
};
```

You can execute this function with a simple call like this:

```
const review = “This was the most delicious fried chicken sandwich I’ve ever had!”;
const sentiment: 'positive' | 'negative' | 'neutral' = await determineSentiment(review);

```

In this case, we should, of course, receive a result of `positive` back from the AI. Function Calling and Typescript allows us to receive this data back from the LLM in a type-safe and structured way, allowing us to pass this data on to additional processes. 

This is an incredibly simple example, but one that can be used to very easily farm actionable information from vast data sets. For example, perhaps `negative` reviews get a ticket created in another system indicating that a customer service representative should follow up with the customer. Or maybe sentiment is tracked longitudinally to show how public sentiment for a particular product or topic trends over time.

A more complex example might be a review like the following:

> Overall, I really love this product and have been using it for years. It's great! However, the new update is terrible and I hate it. I'm not sure if I will continue using it.

There is both positive and negative sentiment present in this statement and with a few adjustments to the LLM prompts we can explicitly tease out sentiment for each category; perhaps in this example we want to know how the latest product update is being received (not well, apparently).

## Wrapping Up
Sentiment Analysis can give you incredibly valuable insights into your business or how the public is feeling about a topic. Getting this data in an automated way is now easier than ever using modern AI tools. Are you sitting on a backlog of customer feedback that is thousands of messages long? Let’s build an AI model that can translate it to actionable feedback for your team. 

The team at Beta Acid is solving all kinds of interesting problems using Machine Learning. What can we help you build?


Sentiment Analysis with GPT-4 Function Calling

### The Tool
Let’s say you want to build an app that enables auto mechanics to inspect a vehicle, verbally dictate their findings, and automatically generate a detailed repair estimate for the customer. This estimate includes pricing for parts and services, which the mechanic can review and adjust as necessary before sharing it with the customer. This app, and many similar use cases, can easily be achieved by using a practical combination of speech-to-text and a generative AI model. Let’s break down how we would go about building this.

### The Technology
The framework of our app is built with standard React Native components, enabling it to operate across iOS and Android devices. It includes a dictation function that records the mechanic’s speech and saves it to an audio file (this is fairly trivial so we’ll skip the explanation of this part).

At the core of our speech transcription process is OpenAI's Whisper, a state-of-the-art model renowned for its exceptional speech-to-text conversion capabilities. Whisper excels in accurately interpreting the mechanic's spoken words, even amidst the backdrop of the bustling noise typical of the auto body shop. This level of accuracy is pivotal in capturing the intricate details of vehicle inspections, ensuring that every observation is accurately documented.

To elevate these transcriptions into actionable auto repair estimates, we harness the power of OpenAI's GPT-4, augmented with [function calling](https://platform.openai.com/docs/guides/function-calling) capabilities. OpenAI’s function calling is the capability to take functions as input and generate structured results.  Our system is backed by a database of services, parts, and their corresponding prices from the shop. 

To get the transcription from an audio file using OpenAI's Whisper is as easy as follows:

```py
import whisper
model = whisper.load_model("base")
def transcribe(audio_file_path):
 result = model.transcribe(audio_file_path)
 return result["text"]
```

When GPT-4 receives the transcription, it doesn't just read the text—it understands it. It delves into the details, distinguishing between different types of repairs and maintenance tasks. By analyzing the context and specifics of the mechanic's notes, GPT-4 can intelligently generate a JSON payload that we then use to create a tailored estimate. The estimate specifies the necessary parts and services, complete with associated costs, directly derived from the mechanic's expert assessment.

In the following code snippet, we show how to call OpenAI’s GPT-4 with a temperature of 0.1.  This indicates we have a model that is not too creative, making it easier to follow our instructions of the initial prompt:

```python
import openai
import os
from dotenv import load_dotenv

load_dotenv()

openai.api_key = os.getenv("OPENAI_API_KEY")

def build_messages(prompt, first_message):
 return [
     {
       "role": "system",
       "content": prompt,
     },
     {
       "role": "user",
       "content": first_message,
     }
 ]

def call_llm(prompt, first_message):
 response = openai.ChatCompletion.create(
   model="gpt-4",
   messages=build_messages(prompt, first_message),
   temperature=0.1
 )
 response_message = response["choices"][0]["message"]
 return response_message
```

The initial prompt, i.e., the initial set of instructions that will guide the chat completion and consequently achieving our desired result:

```
initial_prompt = """
 Based on the following list of auto repair services and/or products, identify in the transcription all the repairs needed and the quantities needed of each. Answer only with the expected json array, no text or root object! Double check if you are including all of the repairs! Think it through, calmly, if you do it successfully, I will tip you $100!


 Response Format (json):
 [
   {{"id": "service_id", "quantity": 1, "reason": "reason why this service is needed"}}
 ]


 Example:
Transcription:
	Starting with the exterior, the car's bodywork is in prime condition. Moving under the hood, the engine is running smoothly with no signs of trouble. However, the brake system requires some attention, likely needing new pads and possibly rotors. The transmission is operating flawlessly, showing no signs of wear or need for repair. Tires are in good shape, with plenty of tread life left, but alignment is slightly off, necessitating an adjustment. The interior is well-kept, with upholstery and dashboard looking like new. The air conditioning system, though, is not cooling efficiently and might need a recharge or a more in-depth look at the compressor. All lights, both interior and exterior, are functioning properly. Lastly, the suspension system is robust, providing a smooth ride without any noticeable issues.
Result:
	[
  {"id": "brake_repair", "quantity": 1, "reason": "Brake pads and possibly rotors need replacing."},
  {"id": "wheel_alignment", "quantity": 1, "reason": "Alignment is slightly off, requiring adjustment."},
  {"id": "ac_service", "quantity": 1, "reason": "Air conditioning not cooling efficiently, may need recharge or compressor inspection."}
]


 Services Available:
 {list_of_services}
 """
```

In the first message sent after the initial prompt, we feed the model with the user transcription so it returns us an JSON with all the services identified on the text, that can consequently be used for automatically calculating the Bill of Services.

### Best Practices for Function Calling Prompts
Getting the best results from GPT-4 function calling is somewhat of an art. In our experience these have worked well:

- **Offer a tip**: We don’t know why, but offering the LLM a $100 tip has yielded better results.
- **Provide examples**: In the prompt, provide a few examples of snippets of transcription and how they should be handled
- **Be specific**: Be as explicit as possible in your prompt for what you’d like the LLM to do.

### The Benefits
Even in its prototype phase, the app has demonstrated the ability to produce accurate estimates more than 75% of the time, promising significant time and cost savings for the business. Additionally, this automation reduces the potential for human error, ensuring customers receive reliable and transparent pricing.

### Opportunities for Enhancements
One promising direction for enhancement lies not in the fine-tuning of models but in the strategic evolution of our interaction with OpenAI's GPT-4 through [chain of thought prompting](https://minimaxir.com/2023/12/chatgpt-structured-data/#unions-and-chain-of-thoughts).

Chain of thought prompting is a technique that guides the language model through a step-by-step reasoning process to arrive at a conclusion or output. Instead of merely asking for a direct answer, this method involves posing a series of logical steps that the model can follow to deduce the answer. This approach is particularly useful for complex queries or tasks that benefit from explicit reasoning, as it helps the model to "show its work," much like a math student solving a problem on a chalkboard.

Incorporating chain of thought into the prompt might look like:
> "Based on the mechanic's dictation, identify and list the specific repair tasks required for the vehicle. Begin by summarizing the mechanic's findings to understand the overall condition of the car. Then, categorize these findings into individual tasks that need to be addressed. For instance, if the mechanic mentions that the brake pads are worn out, list 'replace brake pads' as a task. Similarly, if there's mention of an overdue oil change, add 'perform oil change' to the list. Finally, if a leak in the air conditioning system is noted, include 'repair air conditioning leak' as a necessary task.”

For our app, incorporating chain of thought prompting can significantly enhance the accuracy and depth of the repair estimates generated by GPT-4. By structuring prompts to lead GPT-4 through the mechanic's observations, analyzing each aspect of the vehicle's condition, and then logically determining the necessary repairs and associated costs, we can achieve a more nuanced understanding of the repair needs. This method can also help in identifying ancillary services that might be overlooked by a more straightforward analysis.

### Applications in Other Industries
The potential applications of this suite of technologies extend far beyond auto repairs. Here are a few examples:
- Facility Management in Large Corporate Campuses: Automating maintenance and repair task identification and cost estimation can streamline operations and budgeting for large campuses.
- Quality Control in Manufacturing: Identifying defects and required corrective actions through analysis of inspection reports can improve product quality and operational efficiency.
- Security Guard Building Checks: Transcribing and analyzing security patrol reports to identify and prioritize security concerns and maintenance issues.

### Beyond simple function calling

For use cases where function calling alone might not suffice, integrating a vector database like Pinecone, Chroma, or ElasticSearch can add depth to the analysis. The pairing of a vector database and an LLM is called Retrieval Augmented Generation (RAG). RAG apps offer a sophisticated blend of the generative capabilities of language models with the precise, query-specific information retrieval from vector databases. This synergy enables applications to produce responses that are not only contextually relevant but also deeply informed by the specific data points stored in the vector database.

In practice, RAG can significantly enhance applications in various domains. For instance, in healthcare, a RAG-enhanced system could analyze patient records and medical literature simultaneously to suggest personalized treatment plans. In customer service, it could retrieve and incorporate specific user account information into its responses, providing personalized assistance at scale.

This approach marries the broad, intuitive understanding of language models with the targeted, data-specific insights of vector databases, resulting in outputs that are both accurate and richly informative. As such, RAG applications represent a powerful tool for tackling complex analytical tasks that require a deep dive into large datasets, offering nuanced insights that would be beyond the reach of either technology used in isolation.

### Conclusion

By harnessing the power of advanced LLMs like GPT-4 and cutting-edge transcription technology like Whisper, we're not just innovating in the auto repair industry; we're opening doors to a myriad of applications across different sectors. This blend of AI and domain knowledge stands to redefine efficiency and accuracy in operations and customer service, setting a new benchmark for technological integration in business processes.


Using new machine learning techniques we can use an LLM to automatically estimate car repair costs

Using Machine Learning to Create an Auto Repair Estimate

# Create, Test, and Deploy a Typescript Lambda with CDK

I’ve always wondered why there isn’t a `create-react-app` type template for creating Lambdas, and while we aren’t going to create that now, we’ll go over the basic steps for creating a Typescript-based Lambda, testing it locally, and deploying it to an AWS environment.

If you’re used to manually deploying your Lambdas, CDK may be new to you. Not only are we going to build the container that will store our Lambda handler code, but CDK will also define the infrastructure and help us deploy it all to our AWS account. This will likely be a bit overwhelming (and there’s a lot of boilerplate), but once you get the hang of it things will be much easier than doing things manually.

## Prerequisites

We’ll be using the AWS CLI tools for CDK and SAM. CDK will create a CloudFormation Stack for us which will contain the definition of all the infrastructure we need (eg. a Lambda and an API Gateway). We’ll also use SAM, but only to allow for local testing of our Lambda code.

There are a few things you’ll need to do before we can get started.

- Create an AWS account
- Install the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)
- Setup your AWS credentials with `aws configure`
- Install [CDK CLI](https://docs.aws.amazon.com/cdk/v2/guide/cli.html) (or `npm install -g aws-cdk`)
- Install [SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html)
- Install [Docker](https://docs.docker.com/desktop/)

## Scaffolding

Create the basic template for your infrastructure using CDK.

```
$ mkdir my-lambda
$ cd my-lambda
$ cdk init --language typescript
```

This creates a blank CloudFormation Stack. Open the Stack definition file found at `./lib/my-lambda-stack.ts`. You should find a class called `MyLambdaStack` with an empty constructor. 

Inside this constructor is where we will tell CDK what other resources we need. In our case we will add a Lambda and an API Gateway, but we’ll get to that later.

If you’re curious at this point, you can run the following command to see what resources are in the Stack.

```
$ cdk synth
```

This should spit out YAML code to the terminal that defines everything in our stack. Although it will return a lot of text, it basically doesn’t contain much for infrastructure at this point.

We’ll run this code again later to create a SAM Template, which will be used to synthesize a virtual environment that we can run locally to test our Lambda code.

## Create Your Lambda Handler

If you’re already familiar with Lambdas, this is the easy part. We’re just going to create a Typescript file that exports a Lambda handler. 

We’ll install an NPM package called `aws-lambda` that has a few classes we need and then we’ll create a new folder to store your Lambda code.

```
$ npm install aws-lambda
$ npm install --save-dev @types/aws-lambda
$ mkdir src
$ cd src
$ touch lambda.ts
```

Inside `lambda.ts` we’ll create a basic Lambda handler.

```
import { APIGatewayProxyHandler } from "aws-lambda";

export const handler: APIGatewayProxyHandler = async (event) => {
 const body = {
   message: "Hello from Lambda!",
 };
 return {
   statusCode: 200,
   body: JSON.stringify(body),
 };
};
```

## Transpiling Typescript

Although all the code that was autogenerated for us is in Typescript, there is actually no mechanism currently in place to transpile our Lambda function into Javascript (the Lambda runtime cannot currently interpret Typescript) so we’ll need to do that ourselves.

A basic `tsconfig.json` file was created when we ran `cdk init`, but it's missing a few things. We’ll need to add an `outDir` and an `include` property so the Typescript transpiler (`tsc`) will know what to transpile and where to put it. 

After adding these properties, your `tsconfig.json` should look something like this.

```
{
 "compilerOptions": {
   "target": "ES2020",
   "module": "commonjs",
   "lib": [
     "es2020",
     "dom"
   ],
   "declaration": true,
   "strict": true,
   "noImplicitAny": true,
   "strictNullChecks": true,
   "noImplicitThis": true,
   "alwaysStrict": true,
   "noUnusedLocals": false,
   "noUnusedParameters": false,
   "noImplicitReturns": true,
   "noFallthroughCasesInSwitch": false,
   "inlineSourceMap": true,
   "inlineSources": true,
   "experimentalDecorators": true,
   "strictPropertyInitialization": false,
   "typeRoots": [
     "./node_modules/@types"
   ],
   "outDir": "dist",
 },
 "include": [
   "src/**/*",
 ],
 "exclude": [
   "node_modules",
   "cdk.out"
 ]
}
```

Now run the build command to make sure it worked.

```
$ npm run build
```

This should have created a `./dist` folder that now contains the transpiled version of your Lambda function.

## Updating the CDK Stack

Now that we have our Lambda handler and it’s transpiling to Javascript, we can tell our CDK Stack that it exists.

Go back to the Stack definition file (`./lib/my-lambda-stack.ts`) and replace it with the following.

```
import * as cdk from "aws-cdk-lib";
import { Construct } from "constructs";
import * as lambda from "aws-cdk-lib/aws-lambda";
import * as apigateway from "aws-cdk-lib/aws-apigateway";

export class MyLambdaStack extends cdk.Stack {
 constructor(scope: Construct, id: string, props?: cdk.StackProps) {
   super(scope, id, props);

   // The code that defines your stack goes here
   const handler = new lambda.Function(this, "MyLambdaHandler", {
     runtime: lambda.Runtime.NODEJS_20_X,
     code: lambda.Code.fromAsset("./dist"),
     handler: "lambda.handler",
   });
   const api = new apigateway.RestApi(this, "MyLambdaHandlerApi", {
     restApiName: "My Lambda",
     description: "This is my API for my Lambda",
   });
   const getLambdaIntegration = new apigateway.LambdaIntegration(handler);
   api.root
     .addResource("run-my-function")
     .addMethod("GET", getLambdaIntegration);
 }
}
```

Note: notice that we’ve added an API Gateway that responds to the resource `/run-my-function`. We’ll need this later when we try to invoke our Lambda.

## Running Locally with SAM

Now we can test our Stack and Lambda handler locally.

To do so, we need to autogenerate a `template.yml` from the synthesized CloudFormation Stack so that SAM can read it. We can simply take the output from `cdk synth` and route it to a file.

```
$ cdk synth --no-staging > template.yml
```

Note: Every time we make a change to the Stack we’ll need to update the `template.yml` for SAM. You may want to add a script to your `package.json` to do this.

Because our Stack only contains a single Lambda we can run the following command with no parameters. If you add multiple Lambdas you’ll need to pass in the ID of the Lambda function you want to run.

```
$ sam local invoke
```

Running this should return something similar to the following.

```
$ sam local invoke
Invoking lambda.handler (nodejs20.x)                                                                                                                  
Local image is up-to-date                                                                                                                             
Using local image: public.ecr.aws/lambda/nodejs:20-rapid-x86_64.                                                                                      
                                                                                                                                                      
Mounting /my-lambda/dist as /var/task:ro,delegated, inside runtime container                                      
START RequestId: 786c4a20-2873-4de4-b06a-cdd63fe650cd Version: $LATEST
END RequestId: 4ac6a0bd-df07-419f-9c02-5ce37fc38a2d
REPORT RequestId: 4ac6a0bd-df07-419f-9c02-5ce37fc38a2d	Init Duration: 0.41 ms	Duration: 781.81 ms	Billed Duration: 782 ms	Memory Size: 128 MB	Max Memory Used: 128 MB	
{"statusCode": 200, "body": "{\"message\":\"Hello from Lambda!\"}"}
```

And there you have it! You ran your Lambda locally.

## Deploying to AWS

CDK requires a “bootstrapping” step which installs a CDK CloudFormation Stack in your AWS environment which contains the tools that will be used later when you deploy your new Lambda.

Run the following command and follow the instructions.

```
$ cdk bootstrap
```

Now you’re ready to deploy your Lambda to your AWS environment. All you need to do is tell CDK to do it.

```
$ cdk deploy
```

This will analyze your Stack and deploy the changes to your Stack to your AWS environment. When this completes, you should see the following towards the bottom of your terminal.

```
Outputs:
MyLambdaStack.MyLambdaHandlerApiEndpoint72F57D33 = https://8h6xas3xoy2.execute-api.us-east-1.amazonaws.com/prod/
```

You can copy that URL, append the API Gateway resource from earlier to it, and run it in a browser or using `curl`.

```
$ curl https://8h6xas3xoy2.execute-api.us-east-1.amazonaws.com/prod/run-my-function
{"message":"Hello from Lambda!"}
```

If you’re done with this test and want to delete everything we just created, just tell CDK to destory it.

```
$ cdk destroy
```

## Get Shit Done

Our team has built hundreds of Lambdas over the years and it’s always baffled me how poor the documentation is for setting up something so simple.

What can we help you build?


CDK is a powerful tool for building AWS Lambda functions

Create, Test, and Deploy a Typescript Lambda with CDK

Lucas Sartori

Contextualizing the tools and getting a map plotted with Angular

A Brief Introduction to OpenLayers and Web Mapping

Guilherme Mierzwa

A step-by-step guide to help you migrate content from WordPress to a content management system

Mastering the Art of WordPress to CMS Migration

Daniel Corash

It’s generally acknowledged that handoff can be one of the trickiest aspects of the design process to nail down, and countless articles and tutorials on the topic are a quick Google search away. In my experience however, the vast majority of these are presented from the perspective of designers working at large product companies. In a less structured environment, such as a software development agency, the handoff process can easily become more difficult to navigate, especially when clients are involved. Large companies have processes in place that fit their specific needs, however at an agency, each client brings their own requirements and variables, and like everything else the design handoff process must evolve in real time.

Here’s a scenario that should be familiar to anyone working in such an environment: numerous meetings have been held, designs have been iterated on multiple times, and after months of work on a specific flow or feature, everyone on both sides is aligned. The designs are set and ready for handoff to the development team. They’ve been moved to the appropriate “dev-ready” page inside Figma, and handoff notes and documentation have been added.

After a day or two, the screens are in the process of being built, and everything is going according to plan. Inevitably however, some feature or aspect of the design isn’t going to work, or needs changing. A wrench has been thrown into the process, and suddenly the “dev-ready” screens are no longer ready. Design tweaks or changes are made - this time to the screens that were already handed off to developers. There’s confusion because the engineers don’t know what’s new in the designs, and they don’t have the proper context to understand what changes have been made or how changed features are supposed to work. It’s akin to an architect deciding to make alterations to a building’s blueprints after the foundation concrete has already been poured.

A scenario very similar to this one played out several times during Beta Acid’s recent engagement to rebuild Cardata’s mobile app. In fact, situations like this one are unavoidable. The process of collaborating to build complex digital products is always going to be messy to a certain degree, even if both teams are hardworking, dedicated, and are more or less on the same page (as was the case with Cardata). The key is to anticipate these scenarios and to have structure in place to deal with them.

## Communication is Key

Four main groups were involved in the Cardata design phase: Beta Acid’s designer, Beta Acid’s tech lead and project manager, Cardata’s design team, and Cardata’s leadership team. That means numerous individuals working and referencing designs inside Figma, and it stands to reason that without clear and effective communication between all parties, the project would have been a near impossible task. Overall communication between the teams was excellent, but there was one point Beta Acid could have emphasized to an even greater degree, and that was around the concept of “development ready” screens.

Aligning on the concept of “dev-ready” is key. It’s important to communicate (to both internal and external parties) that when screens are determined to be dev-ready, then they are going to immediately move to development. This doesn’t mean that changes can’t be made (because, again, inevitably they will be) but any additional changes should be thoroughly vetted and of critical importance. Setting expectations around project timeline and the design revision process is crucial so that nobody is surprised when things don’t go exactly as planned.

It’s also important to bring developers into the process at an early stage. Give them insight into design decisions, so that they have an understanding of the product and an idea of what to expect. A beneficial byproduct of this approach is that if something is not possible or difficult to achieve, engineers can raise this as an issue prior to development. 

Further, don’t just handoff designs via Slack - hold handoff specific meetings with developers or project managers, and walkthrough designs and user flows on a call. Answer questions and provide context (time permitting, go one step further and create a prototype to visualize exactly how features are supposed to work and feel). 

Finally, take advantage of features offered by Figma’s new Dev Mode, which go beyond simple screenshots or comments, and include component properties and even code.

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/PkaCj3P7RI2NCUBIGejB)

*Experiment with Dev Mode inside Figma to provide developers with additional context.*

## Organization

As important as communication is organization - it’s crucial to have a consistent file and page structure inside Figma that can adapt when designs change at inopportune moments. There are many different ways to achieve this and there is no one size fits all solution. However, setting up a coherent file structure inside Figma at the beginning of a project is crucial. Depending on the size of your project, multiple files for different sections of an app may be necessary. Breaking a project down into smaller chunks in this manner will lay an effective foundation once handoff comes around. 

Similarly, the importance of organizing and labeling screens and user flows inside files should go without saying. It’s also a good habit to leave notes for developers describing how certain interactions or processes should work. The ultimate goal is for there to be no confusion when you communicate with the development team: “Tracking dashboard mileage submission screens are ready for handoff.”

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/tVP55RbvQOanyHsRqi6C)

*At minimum, designs ready for handoff should have their own page (or potentially their own file) inside Figma, with a clear label and date. Any updates or changes to these screens should happen elsewhere, or at the very least be labeled clearly and relayed to developers.*

## Dealing with Changes

Despite all of these precautions and best practices, changes to “dev-ready” designs will inevitably crop up, and it’s a good idea to have a plan in place when they do. Making changes to dev-ready designs inside Figma after development has already started has the potential to cause numerous headaches for everyone involved. Therefore, it’s a good idea to maintain the initial dev-ready design as a source of truth, and implement future changes on a copy. These can simply live on a separate clearly labeled “design revisions” page inside your design file, or you can go one step further and utilize Figma’s built in version history feature. Either way, new designs should be clearly labeled (before and after screenshots can be helpful in this regard), and in a perfect world, there should be a conversation with developers and other stakeholders to discuss the changes.

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/IhDDAixQyWzhYaSs0zyF)

*This flowchart outlines one potential method for introducing design changes to a source file.*

There are some potential pitfalls to watch out for. For instance, it’s likely that you have been using components, at least for some repeated design elements. This makes it difficult to maintain dev-ready designs in their initial state as a source of truth, because once a component has been updated and library changes have been published, they will update everywhere. How big an issue this is depends on the specific project. Is it really necessary to go back and revisit previous designs? Figma’s version history will likely be helpful in this regard. You might even consider using a separate program like Zeplin to maintain design version control and facilitate the handoff process. Otherwise, simply taking screenshots to document changes along the way may be all that is required. 

## Conclusion

The handoff process can be as complicated or simple as required. It may be an oversimplification, but the truth is that it does really depend on the specific situation. Ultimately it’s incumbent on the designer and the rest of the team to figure out what process is going to work best. But having strong communication, a plan to keep files organized, and a structured approach to deal with changes will establish a foundation for success. 


Navigating the transition from design to development

Design Handoff

Marlon Ruttman

In the ever-evolving world of web development, new tools and frameworks emerge like clockwork. One such innovation that has gained significant attention in recent years is Svelte. If you're already familiar with JavaScript frameworks like React, you might be curious about what makes Svelte unique and whether it's worth exploring. In this article, we'll delve into Svelte, its advantages, disadvantages, and tackle the burning question: Is Svelte really faster than React?

# What is Svelte, and How Does It Differ from React?

Svelte, developed by Rich Harris, is a radical new approach to building user interfaces. Unlike traditional JavaScript frameworks, Svelte takes a different stance on how applications should be built. The most significant difference between Svelte and React is how they handle the DOM and bundle sizes.

- **Bundle Sizes:** React relies on a virtual DOM to update the real DOM efficiently, which can lead to bulky bundle sizes. Svelte, on the other hand, compiles components into highly optimized JavaScript during build time. This means Svelte applications are notably smaller, making them faster to load and more efficient in terms of data transfer.

- **Developer Experience:** React introduced a component-based model to the web, revolutionizing how we build web applications. However, the development process often involves writing a considerable amount of boilerplate code for state management and rendering. Svelte streamlines this process with its intuitive and elegant approach to building components. With Svelte, you write less code and achieve more, simplifying the development experience.

# The Pros and Cons of Svelte



## **Pros:**

- **No Virtual DOM:** Svelte doesn't use a virtual DOM, eliminating the need for complex diffing algorithms. This translates to faster updates and reduced memory consumption, resulting in exceptional performance.
- **Less Boilerplate:** Svelte significantly reduces the amount of code you need to write compared to React. This simplicity leads to cleaner, more maintainable code.
- **Truly Reactive:** Svelte's reactivity system is built-in and effortless. There's no need for useState or useEffect hooks; components react to changes automatically, simplifying the development process.
- **Easier to Learn:** Svelte's straightforward syntax and concepts make it easier for newcomers to grasp. If you're new to web development, Svelte offers a gentle learning curve.

## **Cons:**

- **Smaller Ecosystem:** Svelte's ecosystem is smaller than React's, which means fewer libraries, tools, and community support. You might need to build some functionality from scratch or adapt existing libraries.
- **Unique User Experience:** While Svelte's approach is refreshing, it's not the industry standard. If you work on a team or with an existing codebase using React, transitioning to Svelte might pose challenges.
- **Less Libraries and Tools:** React's extensive ecosystem boasts a wide range of libraries and tools for various use cases. Svelte, being relatively new, has a limited selection of these resources.

# Is Svelte Really Faster than React?

Now, let's tackle the million-dollar question: Is Svelte faster than React? There are several compelling reasons why Svelte is often considered more performant:

- **No Virtual DOM Diffing:** Svelte's absence of a virtual DOM means it doesn't need to diff and patch the DOM, resulting in faster updates.

- **Built-in Shared State:** Svelte offers built-in and highly efficient solutions for shared state management, eliminating the need for external libraries or complex setups.

- **Lightweight:** Svelte is approximately five times smaller than React in terms of bundle size. This translates to faster load times and a more responsive user experience.

- **Reactive by default:** Svelte's reactivity system is inherent, so you don't need to implement hooks like useState and useEffect, reducing the codebase and improving performance.

# Code examples


The code below is enough to have a Hello World. Zero boilerplate.

```
<h1>Hello world!</h1>
```

Defining a state variable and manipulating it with a click on a button.

```
<script>
  let count = 0;
</script>

<button on:click={() => count++}>Counter is {count}</button>
```

With the $: keyword, you can define reactive statements. They work exactly like the useEffect hook in React, but there's no need to import and declare hooks, neither defining an array of dependencies.

```
<script>
  let count = 0;
  $: doubled = `Counter doubled is: ${count * 2}`;
</script>

<p>Counter is {count}</p>
<p>{doubled}</p>
<button on:click={() => count++}>Increase counter</button>
```

This is how you loop through an array. The each block resembles a vanilla foreach, with the addition of the key extractor for each item in parenthesis.

```
<script>
  const items = [
    {id: 1, name: "foo"},
    {id: 2, name: "bar"},
    {id: 3, name: "baz"},
  ];
</script>

{#each items as item (item.id)}
  <p>{item.name}</p>
{/each}
```

Example of conditional logic. It's noticeable that Svelte uses a more verbose and explicit way that can be strange at a first glance, but it's more readable than nested ternary operators.

```
<script>
  let count = 0;
</script>

<button on:click={() => count++}>Counter value: {count}</button>
{#if count > 10}
  <p>big</p>
{:else if count > 5}
  <p>medium</p>
{:else}
  <p>small</p>
{/if}
```

Example of props declaration. Every variable that is preceded by an "export" statement is considered as a prop.

```
<script>
  export let color;
</script>

<p>You picked: {color}</p>

```
Here we import the component above and declare it passing the "color" prop.

```
<script>
  import Nested from './Color.svelte';
</script>

<Nested color="white" />
```

Slots and named slots are a way to pass children to a component. To pass one or more children, you should declare at least one slot on your component. The named slots are used to have more control over placement.

```
<div class="card">
  <header>
    <slot name="telephone" />
    <slot name="company" />
  </header>
  
  <slot />
		
  <footer>
    <slot name="address" />
  </footer>
</div>
```

Here's the import of the Card component, followed by it's declaration.

```
<script>
  import Card from './Card.svelte';
</script>

<main>
  <Card>
    <span>Patrick BATEMAN</span>
    <span>Vice President</span>

    <span slot="telephone">212 555 6342</span>

    <span slot="company">
      Pierce &amp; Pierce
	<small>Mergers and Aquisitions</small>
    </span>
		
    <span slot="address">358 Exchange Place, New York, N.Y. 100099 fax 212 555 6390 telex 10 4534</span>
  </Card>
</main>

```

And this is the final result:
![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/eehtIfYTIe3lFAe643cK)

In conclusion, Svelte's unique approach to web development makes it an attractive choice for many developers, particularly those concerned with performance and bundle sizes. While it may not replace React in every scenario, Svelte's innovative design, faster loading times, and efficient reactivity system make it a compelling option for modern web development projects

# Learning Resources

You can refer to the [official website](https://svelte.dev/) to see the docs and a great tutorial section, where you can learn about all Svelte features.

Svelte has its web framework: [SvelteKit](https://kit.svelte.dev/). It's a fully fledged framework with the same capabilities as NextJS.

If you like benchmarks, [here you can see](https://krausest.github.io/js-framework-benchmark/2020/table_chrome_84.0.4147.89.html) Svelte compared with its competitors.

On the [Svelte Society](https://www.youtube.com/@SvelteSociety) channel on Youtube there's a lot of webinars, tutorials and announcements. It's the best resource to keep informed about Svelte news.

A Refreshing JavaScript Framework

An Introduction to Svelte

Nikki Katsutani

### Where it Began
I love wine. It is an interest and hobby that was passed down to me by my dad. It’s not about having the most expensive bottle from the most prestigious winery in a famous Old World region. What I love is enjoying a tasty beverage that inspires conversation with a group of good people. Wines taste different depending on who you drink it with - and often better with great company. 

This interest has resulted in a tendency to buy a lot of wine. Many wines are meant to age, some of them for 20+ years until they are “ready to drink”. My apartment likely resembles Central Park in the early fall. But instead of squirrels and nuts, I have wine tucked into every nook and cranny that could reasonably store a bottle. 

![Generated using Midjourney - squirrels having a wine party in a forest during fall, preparing for the winter to come.](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/axU7jtfzTYqI3Jgw6PUQ)

### The Idea
My obstacle then became how do I keep track of all of this? I started with a spreadsheet where I logged every bottle including country, region, subregion, designation, grape varietals, vintage, alcohol, residual sugars, purchase price, where I purchased it, etc. I made a map of my apartment, numbered each “wine storage area”, and started including a location code in my handy-dandy spreadsheet. Then I thought to myself, there must be an app for this! 

I browsed through the app store and although there are a number of options, none of them seemed to fit my exact need - most of them geared towards trying to get me to buy even more wine. After some thought and conversations, we decided to build one. 

In my years with Beta Acid, I have watched the design process happen several times as an observer. Picking up tips and tricks of how to think through various flows and best practices of the most common screens. I am by no means a Designer. But I figured I would give it a shot - by hand. 

### What I Learned
![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/olvfHesJTjCbkAwxxoAy)
I gave myself a week to draw up sketches outlining the MVP version of what I needed - an app to track my wine - and decided to call it Vintt. A vault for my vino. I learned a few things that I want to pass on to others that may have an idea for a digital product and do not know the first step to bring it to life.

#### Do not start with the home screen
I started sketching with the perspective of a user and where a typical user starts when they open an app - the home screen. I spent almost my first day drawing (and redrawing, and redrawing) the home screen. My passion for wine took over and I thought it would be so cool to have every idea on the home screen. Next thing I knew, the “home screen” was five screens. Doh! 

Eventually, common sense broke through and I decided to do the home screen last. In its essence, the home screen is a jumping off point for the rest of the app. Like an executive summary, you can’t write it without the rest of the paper. 

#### Too many details
It is so easy to fall down rabbit holes and not even know it. As I thought through the Wine Details screen, I started searching the internet to figure out every single attribute of every single wine in the world. I wanted to capture all types of wine information and account for every scenario.

After a couple hours of this, I realized that the app, especially the MVP, does not need to capture every single little attribute of a wine. For one, no user is going to input every single detail of every bottle they own and the MVP will not have a wine database to pre-fill this information. Secondly, not all of it is relevant when you are trying to decide what wine to drink with dinner. 

I erased half of my sketch and settled on the top five attributes. I reminded myself for the rest of the process to take a step back often and think through what I, as a user, really needed in this app.

#### A short deadline
For a few days, I kept going back to my sketches to change and tweak things that I thought were not quite right. If I would have compared my drawings from 3 days ago to today, I guarantee you could not spot any significant differences. I’m glad I gave myself only a week as I’m sure I could have gone a month without feeling like I was done. 

I know that it is important to build a proof of concept that I can put in my hands and use in real life. It will be more valuable to define the product than these small tweaks. Even armed with this knowledge beforehand, I was eager to keep refining my sketches. Deadlines are important, if only to keep us from over thinking every step of the way. 

### Summary
I hope my personal experience and learnings will help an ideator out there pause thinking and start building. Over the next few weeks, we will build out a proof of concept of Vintt and share updates throughout the process. 

Let’s build something together! 

Taking the first step from idea to product

Wine & Design

The Secret to Frontend-Backend Coordination

Orval

In software development projects, especially when separate teams handle frontend and backend systems, coordination can be a major problem. Misunderstandings about data structures, error handling, and other API behaviors can lead to bugs, delays, increased costs, and decreased product quality. These misunderstandings are not minor inconveniences; they can derail a project's timeline and lead to friction between teams.

At Beta Acid, we do a lot of these types of projects and often build REST and GraphQL APIs. Both of these formats can benefit from the principles outlined in this article but we will be specifically discussing tools and processes for REST APIs. 

### API Contracts Can Help

To address these challenges, API Contracts have become increasingly important. An API Contract is a comprehensive document that outlines the specific behaviors, data structures, and other aspects of each API endpoint of a project. It serves as a single source of truth that both frontend and backend teams can refer to, ensuring that everyone is aligned in their expectations and implementations. The [OpenAPI Specification](https://swagger.io/specification/) is typically the standard used to create these contracts.

### Tools for Creating API Contracts 

Manual creation of an API Contract is prone to errors and time-consuming. Various tools have been developed to streamline this process. [Swagger Hub](https://swagger.io/) is one such tool that has been widely adopted. However, we've recently transitioned to using [Stoplight.io](https://stoplight.io/) for our projects. Stoplight.io offers a graphical user interface that simplifies the creation of API Contracts and minimizes the risk of errors. Although the tool has its limitations and occasional bugs, the overall efficiency gains make it a worthwhile investment.

### Leveraging Orval for Frontend Development

At Beta Acid, we've taken our efficiency a step further by using [Orval](https://orval.dev/) in conjunction with our API Contracts. Orval is a tool that automatically generates the front end networking code (React Query in this case) based on an API Contract. This significantly reduces the workload on frontend developers, allowing them to focus on building out features rather than writing boilerplate code for API calls. 

We’ve found this tool to be so useful that we are both sponsors and open source contributors. 

By integrating Orval into our workflow, we've been able to:

- **Accelerate Development**: Frontend developers can immediately start working with auto-generated React Query hooks, speeding up the development process. We’ve found that this can increase delivery time by up to 25%.
- **Reduce Errors**: Automatically generated code minimizes the risk of human error, leading to more robust applications.
- **Maintain Consistency**: Using Orval ensures that the frontend code is always in sync with the API Contract, making it easier to manage changes and updates.

### A Practical Example: The Swagger Petstore API Contract

To better understand the utility of API Contracts, let's delve into a concrete example using the OpenAPI Specification for the canonical "[Swagger Petstore](https://petstore.swagger.io/)" application. This example will help illustrate how an API Contract serves as a blueprint for both frontend and backend teams.

```
openapi: "3.0.0"
info:
  version: 1.0.0
  title: Swagger Petstore
  license:
    name: MIT
servers:
  - url: http://petstore.swagger.io/v1
```

This initial section sets the stage by providing basic information about the API, such as its version and title. It also specifies the base URL where the API can be accessed. This foundational information gives both frontend and backend teams a common starting point.

```
paths:
  /pets:
    get:
      summary: List all pets
      operationId: listPets
      tags:
        - pets
      parameters:
        - name: limit
          in: query
          description: How many items to return at one time (max 100)
          required: false
          schema:
            type: integer
            maximum: 100
            format: int32
```

Here, the API Contract outlines the available paths and operations. For example, the `GET /pets` operation is clearly defined, including optional parameters like limiting the number of pets returned. This level of detail ensures that frontend developers know exactly what data they can request, and backend developers know what they need to provide.

```
responses:
  '200':
    description: A paged array of pets
    headers:
      x-next:
        description: A link to the next page of responses
        schema:
          type: string
    content:
      application/json:
        schema:
          $ref: "#/components/schemas/Pets"
  default:
    description: unexpected error
    content:
      application/json:
        schema:
          $ref: "#/components/schemas/Error"
```

This part of the contract specifies what kind of responses to expect. For instance, a successful request to list pets will return a paged array of pets, and may include a link to the next set of pets if pagination is used. This helps both teams understand what the data will look like and how to handle it in their respective systems.

```
components:
  schemas:
    Pet:
      type: object
      required:
        - id
        - name
      properties:
        id:
          type: integer
          format: int64
        name:
          type: string
        tag:
          type: string
```

Finally, the contract defines the structure of the data, specifying what fields are required and what types they should be. This eliminates guesswork and ensures that both teams are aligned in their understanding of the data structures they will be working with.

### Extending the Utility of API Contracts with Internal Testing Tools
We’ve taken this a step further by developing an internal testing tool to verify that the APIs we develop match up to the API contract. This Python-based tool leverages the OpenAPI Specification to generate a test plan and execute tests, ensuring that the API behaves as expected.

#### How It Works
The tool reads the OpenAPI JSON specification from a given URL and generates a structure for all available endpoints. It identifies the allowed methods, parameters, and request bodies for each endpoint. This information is then used to create a test plan that can be executed to validate the API's behavior against the contract. This can then be used as a pre-commit hook in your Github workflow. 


### Alignment with Beta Acid Principles

The use of an API Contract aligns perfectly with Beta Acid's core principles:
- **Always Be Delivering**: API Contracts enable parallel development efforts, allowing both frontend and backend teams to work simultaneously toward project completion.
- **Be Fast, Keep Focused, Stay Flexible**: With a well-defined API Contract, the development process becomes more streamlined, enabling rapid iterations and the flexibility to adapt to changing requirements.
- **Communicate Clearly**: The API Contract serves as a clear, unambiguous form of communication between the frontend and backend teams, ensuring that everyone is aligned in their understanding of the project's technical aspects.

### Summary 
Development projects are often presented with the perfect storm: separate backend and frontend teams, building in parallel, on a very tight timeline. Limit your risk, up your communication, and gain efficiency by leveraging API contracts supported by key tools like stoplight.io and Orval to Get Shit Done. 

What can we build together?

Bridging the Gap Between Frontend and Backend Development

API Contracts

_We love building things._

If you’ve ever met me or heard me talk about why I started Beta Acid you’ve likely heard me say that I love building cool shit. It's why I do this job and it's a passion everyone on our team shares. Most of us have been doing it for a long time and we’ve gotten pretty good at it. 

So when people ask me if we're an Agile team, I get excited. Not because I think Agile is great, but because I think it's mostly a waste of time. Something that is used by managers to make them feel like they are in control, oblivious to the fact that most of the team finds the process tedious, the ceremonies unproductive, and the meetings fatiguing.

So, no, we are not an Agile team.  But we do some Agile-ish things.

It’s important to understand that Beta Acid is not a product company; we are a software delivery company. This means that we don’t adhere to many of the traditional tenets that many other software companies do. Although there are plenty of dreamers on our team, and we’re happy to help you dream, ultimately we are hired to get shit done.

So what is it like to build software with the team at Beta Acid? 

### Principles

There is a core set of values that drive what we do. Most of these are principles that come from our extensive experience building products in the startup world; a cutthroat place where only those who Get Shit Done survive.

- **Always Be Delivering** – everything we do should be in support of bringing a project closer to completion.
- **Be Fast, Keep Focused, Stay Flexible** – get things in front of customers, users, or clients as quickly as possible and be ready to iterate as priorities change.
- **Good is Better than Perfect** – perfection is a distraction that gets in the way of getting good things done.
- **Communicate Clearly** – proactively ensure that everyone is clear on status, goals, and priorities.
- **Better Together** – every person on the team is there for a reason and our outcomes will always be better when we work together.


Adherence to these principles leads to greater productivity and increased efficiency, and has an awesome side effect of people being happier and more fulfilled with their work (or at least that's true for our team). No one on our team complains about having too many meetings or not having time to get work done.

### Method

As you'd expect, you won’t find hardcore Agile Advocates or Scrum Masters on our team. Instead, we pull in processes that work for us from a variety of existing frameworks (Agile, RUP, Spiral, Waterfall, etc.) and tweak them to suit our needs. Every project we work on has different needs and every team we work with has different communication styles, so we focus on being adaptable to these needs while maintaining a few key tenets.

Our workflow for a new project contains four phases:

- **Discover** – we dig in to understand a project’s goals and scope.
- **Design** – we extrapolate from our initial discovery to design and architect the solution.
- **Develop** – we build and test through an iterative process that is receptive to changing priorities while ever marching towards completion.
- **Deploy** – we create infrastructure and operational processes for launching products in the market.

Generally, we focus on bringing net new software products from conception to delivery and our methodology reflects that. We approach improving and maintaining existing products in a different way and would be more likely to take a traditional approach for those projects.

### Team

Arguably, the factor that will contribute most to the successful delivery of a product is the team that is responsible for building it. 

Our product is our people and our people are awesome. We’ve assembled a team of doers (and dreamers) who are incredibly experienced in their fields. There is no need for egos or politics when everyone you work with is equally talented, thoughtful, mature, and kind. 

We are all entrepreneurs, creatives, engineers, organizers, doers, and dreamers that work together to get shit done.

### Outcome 

Through this lens, we are able to deliver an outstanding product, effectively and efficiently, and have fun doing it. Instead of boxing ourselves into a prescribed framework, we pick and choose the best of what works for us and the teams we work with.

Let's build some cool shit together!

We love building cool shit. Here's how we do it.

How Beta Acid Builds Software

We’ve been using Stripe since it first launched over 10 years ago. Stripe has always been one of the easiest ways to take payments online but over the years they have incrementally improved their platform and made things easier and easier. 

When we needed to add a paid, premium tier to [Pourwall](https://pourwall.com), we knew it would be a great chance to take advantage of [Stripe Checkout](https://stripe.com/docs/billing/subscriptions/build-subscriptions). Stripe Checkout has a Customer Portal that takes care of all sorts of things for your customers, like being able to subscribe, unsubscribe, change payment plans, update credit card info, view old invoices, and more.

Here’s what the Customer Portal looks like in Pourwall.

![Stripe Customer Portal](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/GG1pg2TlTJW37KjvBQc4)

Similar to many of the products the Beta Acid team develops for our clients, Pourwall is built on React.js and has a Node.js back end (and we use Typescript for both). Here I’ll give you a quick walkthrough of how to easily add subscription payments to your own React/Node SAAS platform.

For the sake of brevity, I’m going to assume you already have a Stripe account and have created a Product with a single Price that can be used as your recurring subscription cost. You can find explicit instructions on how to set this up in the [Stripe docs](https://stripe.com/docs/billing/subscriptions/build-subscriptions?ui=checkout#create-pricing-model). We’ll use this later so you’ll need to save the Price ID from this screen.

![Stripe Product Pricing](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/PPngnTimROWXFHRZr7HA)

For Pourwall, the basic flow we need to build is as follows:

1. In our app’s front end, we need a button to allow the user to sign up for a subscription plan. That button issues a call to our API and the API returns a URL that we redirect the user to.
2. On the API side, when we receive the request from the button on the front end, we create a Customer record in our Stripe account (save the customer ID to our app’s database), and then initiate a Stripe Customer Portal Session.
3. At this point the user will be redirected to a URL at billing.stripe.com and we are no longer in control.
4. When the user is done, Stripe will immediately send a request to a webhook that we create in our API which includes the details of what the user did (successfully signed up, what plan they subscribed to, or if they canceled out of the screen without doing anything). We use this information to update the user’s record in our database with their subscription status.

Practically speaking, there will be some edge cases you’ll want to handle and a few little tweaks you’ll want to make depending on the experience you want your users to have, but that’s really about it.

Here is our React component with the button that calls our API endpoint. When the API returns the `stripeCheckoutUrl` we redirect the browser to Stripe to finish off the rest.

```
export default function AccountSettings() {
  const handleCreateSubscription = async () => {
    const { stripeCheckoutUrl } = await fetch("/api/create-subscription");
    if (stripeCheckoutUrl) {
      window.location.href = stripeCheckoutUrl;
    }
  };

  return (
    <Layout>
      <div className="max-w-3xl mx-auto p-8 lg:p-2 lg:pb-24">
        <div className="flex flex-row justify-start">
          <Button
            variant="contained"
            color="primary"
            onClick={handleCreateSubscription}
          >
            Upgrade
          </Button>
        </div>
      </div>
    </Layout>
  );
}

```
On the back end, we have a couple simple functions to create a customer in the Stripe platform and create the Customer Portal session. These are simply called from the API endpoint at `/api/create-subscription`. This assumes you have the `stripe` package installed and that you have environment variables with your Stripe keys.

In these examples, we pass in an `Account` object, which is just an object from our database that represents an account.

```
export const createCustomer = async (
  account: Account
): Promise<stripe.Customer> => {
  const stripeClient = getClient();
  const stripeCustomer = await stripeClient.customers.create({
    description: `${account.name} created by ${account.email}.`,
    email: account.email,
    name: account.name,
  });
  return stripeCustomer;
};
```

In the following snippet we also pass in a `priceId` parameter, which comes from the Pricing section of the Product you created in Stripe that we talked about earlier.

```
export const createCheckoutSession = async (
  account: Account,
  priceId: string
): Promise<stripe.Checkout.Session> => {
  const stripeClient = getClient();
  const session = await stripeClient.checkout.sessions.create({
    billing_address_collection: "auto",
    line_items: [
      {
        price: priceId,
        quantity: 1,
      },
    ],
    mode: "subscription",
    success_url: `${APP_BASE_URL}/account?success=true`,
    cancel_url: `${APP_BASE_URL}/account/subscription?canceled=true`,
    customer: account.stripeCustomerId,
  });

  return session;
};

```

This essentially takes care of everything you need to collect credit card information and subscribe your user to a subscription plan. 

Presumably you will also want to reflect this in your app by unlocking certain features once the subscription is active. To do so, you’ll need to create another API endpoint that acts as a webhook that Stripe can call whenever something changes (the customer first subscribes, changes their plan, their credit card is declined, etc.)

This step is a little complicated because we want to verify that the request coming to the webhook endpoint is actually from Stripe. We do this by validating the `stripe-signature` HTTP header with our Stripe secret key. If you skip this, anyone who discovers the URL of this endpoint could issue requests to it and possibly get themselves a free subscription or delete someone else’s subscription.

We use Koa but the following code should work on Express, Next.js, and any other back end framework with very few changes.

```
export const handleStripeWebhook = async (ctx: Context): Promise<void> => {
  const signature = ctx.request.headers["stripe-signature"];
  if (!signature) {
    console.log(`No Stripe signature found in request headers.`);
    return ctx.sendStatus(500);
  }

  try {
    const stripeClient = new stripe(STRIPE_SECRET_KEY, {
      apiVersion: STRIPE_API_VERSION,
    });
    const event = stripeClient.webhooks.constructEvent(
      ctx.request.rawBody,
      signature,
      STRIPE_WEBHOOK_SECRET
    );
    if (event.type === "customer.subscription.created") {
      await createSubscription(event);
    } else if (event.type === "customer.subscription.updated") {
      await updateSubscription(event);
    } else if (event.type === "customer.subscription.deleted") {
      await deleteSubscription(event);
    }
  } catch (err) {
    console.log(`Webhook signature verification failed.`, err.message);
    return ctx.sendStatus(400);
  }

  ctx.body = "OK";
};

```

The functions you see here (like `createSubscription`) are simply blocks of code specific to our app that do things like update database records with the status of the user’s subscription.

And that’s it. That’s all it takes to add Stripe subscriptions to a SAAS app 🍻

How we can leverage Stripe Checkout and Customer Portal to easily build subscription management into a React.js and Node.js app.

Easily Add Subscription Payments to your SAAS Product with Stripe

We’re kicking off our 4th season of [MusicLeague](https://musicleague.com) over here at Beta Acid and I thought it would be a good opportunity to talk about how we use the music competition platform to enjoy time with each other.

If you’re not familiar with MusicLeague, I’ll give you a recap of how we play. Each season lasts about 8 weeks. Each week, the MC of the league picks a theme and everyone that’s playing that season has to submit a song that they think matches the theme. Here’s a recap of some of the themes we’ve had in previous seasons (and which song won that round):

- **Earworms**: Songs that you can't get out of your head.
    - **Winner**: Luis Fonsi - Despacito
- **Worst Song Ever**: Submit the worst. Vote for the worst.
    - **Winner**: Rebecca Black - Friday
- **It's a rainbow 🌈**: Song titles must have a color in them.
    - **Winner**: Rolling Stones - Paint it Black
- **Location**: Songs that have a location (city, country, landmark) in the title
    - **Winner**: DJ Jazzy Jeff & The Fresh Prince - The Fresh Prince of Bel-Air
- **1980s**: Songs released from 1980-1989
    - **Winner**: Eurythmics - Sweet Dreams (Are Made of This)


There’s strategy involved, though, because it’s not just about what song **you** think is best, you have to pick a song that everyone else will think is the best, too.

Through the various themes, we've learned unexpected facts about our colleagues. Who would have guessed that our quiet back end engineer was a hardcore metal fan? Or that a surprising number of people love Black Pink? These revelations spark conversations outside of the game, allowing us to connect with each other on a personal level.

MusicLeague is a platform where titles and positions don't matter and each song is submitted anonymously, the submitter is only revealed after voting is complete. It doesn’t matter if you’re an engineer, a designer, a project manager, or the CEO (although the CEO has been accused of cheating multiple times). This leveling of the playing field fosters a sense of unity and ensures that every voice is valued.

With each song submission, team members have a chance to share and discuss the reason behind their choice. These anecdotes, ranging from heartwarming to hilariously absurd, provide a glimpse into each person's life, bridging the gap between professional acquaintances and genuine friends.

At Beta Acid, MusicLeague has become more than just a game; it's an avenue for strengthening relationships, understanding diverse perspectives, and fostering a cohesive team culture. As we continue to grow and evolve, MusicLeague will be there with us. It isn't just a game for us; it's the unofficial glue holding our wacky, wonderful team together. Rock on! 🎸🤘


The Different Ways We Enjoy MusicLeague

Triumphs and Trolling

### Summary
Flutter and React Native are the two most popular frameworks for creating cross platform mobile applications. Both are excellent choices when you need to develop one application that needs to run on both iOS and Android devices. 

You can create a high quality, high performance app for most use cases using either. The most important issues to consider when making the decision of which technology to use are:
the skill set of your current team
your ability to hire engineers for the project
the developer experience of each platform

### Biggest differences
Technically, the biggest difference between the two is how each renders the user interface. 

Flutter ‘draws’ the components of the app onto the screen directly. You will primarily have the same look and feel across platforms (with some differences to make the components conform to expectations of each operating system). When the next version of an operating system is released, the look and feel won’t change.. 

React Native transforms the components written by a developer into native UI components. This results in more familiar looking components to the users of that operating system. When the user updates their operating system the UI of the app will automatically reflect the changes.

The second important difference between them is that Flutter uses the Dart programming language while React Native uses JavaScript or TypeScript. TypeScript and Dart are close enough in performance and usability that there’s not a clear winner between them. Where it matters is the depth of the talent pool. JavaScript/TypeScript have a large advantage. Dart is not a particularly hard language to learn, but it is certainly a factor.

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/LUviJWAhTDiK1LiTmPZn)
> A Stack Overflow survey of the most popular technologies ([source](https://survey.stackoverflow.co/2022/#most-popular-technologies-language))


### Performance
For most use cases, the performance between the Flutter and React Native should be negligible. Both platforms have recently made significant improvements in this respect, React Native with its [JavaScript Interface](https://github.com/facebook/react-native/blob/main/ReactCommon/jsi/jsi/jsi.cpp) and Flutter’s recent switch to the [Impeller Rendering Engine](https://docs.flutter.dev/perf/impeller) for iOS.

If eking out every bit of performance is important to your application, then you’ll want to avoid using cross-platform frameworks and develop your apps natively in Swift and Kotlin.

### Developer Experience
Availability of 3rd party packages for React Native is larger because it draws on the NPM ecosystem, but Flutter’s is catching up and has 3rd party packages for most important use cases. 

Debugging is easier and more tightly integrated in Flutter than it is in React Native.

Flutter was ranked much higher in Stack Overflow’s 2022 Loved vs Dreaded survey

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/zTgd8MiaTVMgGZyliOfh)


### Web and desktop applications
Flutter has the ability to build web and desktop applications from the same codebase as your mobile application. Both are relatively new, but considered stable by Google. There are limitations on what you can do for web and desktop applications in Flutter, and not all 3rd party packages support them. 

React Native does not currently have production ready web or desktop applications, but they do have some experimental offshoots.

### Conclusion
The mobile teams at Beta Acid have extensive experience working with both Flutter and React Native. We are confident that either choice will be great for our clients technical needs. We make it a point to carefully talk through the softer issues around our client’s current skill sets, developer experience, and ability to hire before making any recommendations. 


Flutter vs React Native: Practical comparison

Wow! That design process was a doozy. But you've made it! Time to kick off development! But… Now what? How do you translate your cutting edge and beautiful designs into concrete tasks for your development team? 

Here at Beta Acid, we specialize in doing this very thing. Here are some helpful tips on how we approach writing User Stories for front-end development. This approach can be used for all types of front-end projects including responsive web apps, native mobile apps, etc. 

### Tips Before Starting

1. We don't care about any fancy methodologies or frameworks (gherkin is only a type of cucumber to us). The point is communication so use whatever works best for your team. 
2. Designs are never final so don't assume that they are. Designs tend to continue to evolve even after they are approved. Select an approved version and use that as a reference for your story. Plan for adopting changes through an iterative development process. 
3. Don't assume that something is "standard" and will be built by default. Unless explicitly documented, you risk the chance of something being missed.

### Purpose of User Stories
The purpose of a well written User Story is to communicate design and product requirements for the piece of functionality the Engineers are building. It should be the single source of truth for Engineering during the development process.

User Stories should break down designs into bite-sized pieces of development that can be tackled by a single engineer. If it is helpful for your team, you may also want to consider limiting User Stories to, at most, one day of development time for a single engineer. This prevents User Stories from getting too large/complex which could impact both development and QA. 

User stories are also heavily used by Quality Assurance teams. A well written User Story can be used as a checklist of what to test and how to test it. It sets expectations with your QA team and helps to ensure a smooth and efficient QA process. 

### Who Should Write User Stories 
Anyone with adequate knowledge of the product and design requirements can write User Stories for development. It is typically the responsibility of a Product Owner or Product Manager. However, writing User Stories can also be completed by a Project Manager or Engineer with the proper knowledge and background. 

### Key Elements of a Well Written User Story

#### Screenshot of the screen from your designs 
You can also supplement the screenshot with a link to the full design file. However, providing only a link to the design is not recommended. A screenshot draws a line in the sand of what the engineering team will build. When the designs change (and they will), there will be a misalignment between the written User Story and the design file. You also lose the history of what design was used to write the User Story. With a screenshot of the designs in your User Stories, you create a single source of truth for what is to be built and retain history which will reduce the risk of a confused Engineering team.

#### Content and Functionality
For simple screens or features, look at the screen designs from top to bottom. As you're scanning the screen, write down everything that you see and what it does. It is important to be as explicit, detailed, and prescriptive as possible. In conjunction with the design screenshot, the developer should have everything they need within the User Story to understand exactly how the screen should appear, what functionality it should have, and exactly what  happens when the user interacts with any part of the screen. For more complex features, use the same process as above but it may be appropriate to break the features down into smaller stories.

#### Don't forget about:

- User States. Does the functionality or screen change based on the User's status, role, or other attribute? For example, the User is logged in vs. logged out, the User is an administrator, the User has completed a particular task, etc.
- Validation. What information should be validated or checked? If the information is incorrect, what should happen? Consider complex validation scenarios as well; does one field depend on another?
- Loading States. What does the screen, its contents, or CTAs look like when loading occurs?
- Empty States. Is there a situation where a list of data or other information may not be present? If so, how should the screen handle it?
- Hover States. If the user hovers over a component with their mouse, what should happen? 
- Error States. What should happen when an error occurs? 
- Analytics. What analytics should be captured on the screen?
- Keyboard Types. For mobile apps, if a keyboard pops up for an input field, what sort of keyboard should be displayed? Should it have a "Return", "Next", or "Go" button? Should it be restricted or are all characters allowed?
- Animations. Does something on the screen automatically move? If so, describe exactly how it should move. Or, include the animation file that should be used for development. 

### An Example
Below is a real-life example of a story we wrote for the Login Screen of a mobile app build.

<div style="background-color:#f0f5fa;padding:8px 24px;">

#### Header
"Back" CTA to the left. When tapped the user should get to the previous onboarding screen.
ABC Company logo on the right. When tapped, the user is directed to the Home screen.

#### Screen Contents
Screen will show a "Login" title. 
A text input field with the title "Username"
- When the user taps into the field, the default keyboard should appear. 
- On the default keyboard "returnKeyType" should be "next". When tapped the user will lose focus of this field and focus will move to the next field "Password".
A text input field with the title "Password"
- When the user taps into the field, the default keyboard should appear. 
- On the default keyboard "returnKeyType" should be "done". When tapped the user will lose focus of this field and the keyboard disappears.
- Password text input field will be obfuscated by default and show an open Eye Icon. 
- If the user taps the open Eye Icon
        - Change to a closed Eye Icon
        - Display the password as plain text (not obfuscated)
- If the user taps on the closed Eye Icon 
        - Change to an open Eye Icon 
        - Obfuscate the password
A "Sign In" primary CTA
- The CTA is always in an active state
- When tapped it will make a call to the Login endpoint and the loading indicator will appear.
- Client side validation:
        - If the user taps "Sign in" but the username and/or password text input fields are empty. 
                - Display error messages under each empty field: 
                - Username: "Please enter a username" 
                - Password: "Please enter a password".
                - Each empty field will appear outlined in red.
- Endpoint responses:
    - [URL or relative path of the endpoint - removed for confidentiality]
    - 200 response
        - if (mfa_setup_required = true) navigate to MFA screen
        - else: store the 'token' in secure storage and navigate to the dashboard screen.
    - 400 response
        - Display 'message' value from response

#### Footer
- "Privacy Policy" CTA at the bottom right side of the screen. 
- When a user taps on it, they will be able to see the Privacy Policy, shown in an in-app browser.

#### Analytics
- Record a Page View event
- Field validation errors for username and password fields
- Tapping the "Sign In" CTA

</div>

### Pro Tips

##### Use Component Names
If you are using a design library or design system, use the actual name of a component in your User Story. This will ensure that the correct component is used.

##### Capture All Error States
Work with your Engineering team to get a list of all errors that can occur on a particular screen. Make sure your User Story documents all possible errors in order to ensure that every scenario is handled gracefully. 

##### Order your Epics for Development
The order in which you tackled the design process may not translate well for development. Have a chat with your development team to sequence your Epics for development in the most logical way possible. Often, the first three epics are:
- Design library and common components: build it once then reuse them over and over again. It will speed up the rest of development.
- Global navigation: so you have a way to navigate through your app to different areas of functionality
- User creation and authentication: in many apps, you can't do much else if you don't have a user to do it with. 
- After that, it really depends on your product so rely on your engineering team for guidance. 

#### Timing of Back-end Development
The following discussion of team sequencing probably deserves an entire article of its own. Where possible, we try to keep the back-end team two epics ahead of the front end team; sometimes this means doing back-end work while designs are still in progress. This allows the front-end team to hit the ground running as soon as designs are approved.

Yes, you can use mock endpoints in a pinch but this approach adds two phases of work to your schedule: building mock endpoints and re-testing against real endpoints when they're ready. It also often causes a bunch of re-work if your API contracts are not solid (thus your real endpoints end up being different from the mock endpoints you built).

### Is it worth it?
A well written User Story does take (a lot of) time. Sometimes, especially for very small, close knit teams that have worked together in the past with solid relationships between product, design, and engineering, the level of detail in a well written User Story is not worth the amount of time they take to write. However, there are many scenarios where well written User Stories are highly valuable. A couple scenarios are:
Large teams where there might not be very much communication between a product owner or designer and the engineers, 

Teams where new engineers are being onboarded mid-project and lack historical or domain knowledge.
In these cases, a well written story is your go-to tool to communicate requirements so the detail is crucial. Evaluate where your team is on this spectrum and adjust accordingly. 

### Conclusion
This is how we do things at Beta Acid, but it's not a one-size-fits-all solution. We believe adaptability and flexibility is critically important. For every project we evaluate how to be as effective as possible given the variety of client and team needs. Sometimes, a well written User Story creates efficiencies that far outweigh the time commitment to write them. 

Want to learn more about working with us? Reach out to: <a href="mailto:hi@betaacid.co" style="color:#B400FF;font-weight:600;">hi@betaacid.co</a>.

Writing User Stories for Front-End Development

Fábio Colombo

A serverless architecture allows you to configure your system once and avoid most of the maintenance and operational work. Serverless architectures will generally scale well and without any operator interaction. Most serverless architectures are based on cloud services priced based on how much you use the service. AWS Lambda, API Gateway, CloudFront, and S3 are examples of this. AWS Lambda pricing is a product of the run time of your function, i.e. how long it takes to load and run and the amount of memory and disk used by the function. API Gateway and CloudFront pricing are both the sum of requests handled and data transferred. S3 pricing depends on the amount of data stored (as well as how this data is stored) and the amount of data transferred.

This pricing model is great for projects just starting out since they provide a robust infrastructure that requires little setup and maintenance while likely costing less than running a traditional infrastructure based on your own VMs or on-premise.  Although serverless architectures require less setup and maintenance, it is still important to check if your costs are scaling well with increasing traffic. In this blog post, we will look at a cost reduction analysis for one of our projects and how you can apply this to your own projects.

### Analyzing your Bill

Cost analysis always starts with the bill. AWS, GCP, Azure, and other cloud providers have services specifically for cost analysis. AWS provides the [Cost Explorer](https://console.aws.amazon.com/cost-management/home) dashboard, which allows you to filter your costs by service, such as Lambda, API Gateway, S3, etc., and can also project month-end costs based on your previous usage. Google Cloud has a similar feature in [Cloud Billing](https://cloud.google.com/billing/docs), as does [Azure](https://learn.microsoft.com/en-us/azure/cost-management-billing/) and many other cloud service providers. Learning to use the features available for your provider is an important step to understanding and optimizing your cloud infrastructure costs.

Another important aspect of cost analysis is how you plan to distribute costs among your projects. By using a different AWS Account (or GCP project) for each project, you can have a clearer picture of costs per project. AWS allows you to have linked sub-accounts so that billing is centralized in a single account, but resources can be split up among different accounts. This is great for cost analysis as well as for security, as credentials can be managed with greater ease in each sub-account. You can give a team greater control over their specific account or project without having to worry about them accidentally messing up infrastructure managed by a different team.

If creating new accounts or projects is not an option, you still have options for splitting up costs. AWS allows you to use [special tags](https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/cost-alloc-tags.html) for cost allocation. GCP allows you to [add labels to cloud resources](https://cloud.google.com/blog/topics/cost-management/use-labels-to-gain-visibility-into-gcp-resource-usage-and-spending) so that you can later filter costs using these labels. Depending on your cloud infrastructure, you may need to modify your code to use logs or register metrics to allocate costs.The AWS, Azure, and GCP cost management features also allow you to create budgets and automated alerts to inform you beforehand that you are spending more than you expected, giving you time to take action to reduce your costs.

Although learning about the newest technologies may seem more exciting than learning how to navigate and query cost data, this is an essential skill if you want to optimize your infrastructure. Knowing how each cloud service is priced will help you better plan the correct infrastructure for each use case. Knowing how to analyze costs will allow you to focus your optimization efforts on the services that make up the largest percentages of cloud costs. Like in software optimization, the final savings will be a product of the percentage saved by the service and the percentage of the cost made up by that service.
 
> *Savings = Service cost percentage * Service cost savings*

### Pourwall
This blog post will center on our analysis of [Pourwall](https://pourwall.com/). Pourwall is a simple and free digital menu system for bars and breweries. It comprises a landing page to inform users about what Pourwall does, a web app to configure menu items and how they are displayed, and an app that runs on TVs to display the menus to customers.

Pourwall runs on cloud infrastructure hosted on AWS. We use S3 to host the website content, such as HTML, CSS, and Javascript files and images. CloudFront is used to distribute this content, RDS to host our database for storing user data, Lambda functions for our backend API, and Cloudwatch for logging. RDS has a fixed monthly cost based on the instance size. Everything else is charged by usage. The more requests we receive, the more we have to pay. 
Pourwall costs were above expected for the number of visitors and devices we had connected. We used Cost Explorer to analyze our AWS service usage and costs and determine which AWS services we should focus our efforts on to obtain a cost reduction.

This is easy in Cost Explorer, as it can group our costs by service. The graph below shows how much we spent per day during the previous month on the various AWS services we use.

![Cost breakdown by AWS service](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/zN2DelHIRHCaahA7tygF)

Analyzing this graph, we can easily see that the API Gateway service should be a priority. If we were to save 50% on our API Gateway costs, which make up 49% of our monthly costs, we would lower our final bill by 24.5%. On the other hand, If we lower the Cloudwatch costs by 50%, our final bill would only be 4.5% smaller since Cloudwatch is only 10% of our final bill. This analysis should be redone as you make improvements, as some changes can impact more than one cloud service, making optimizations to other services more worthwhile.

Looking at our breakdown, we see that most of our cost is due to the large number of API requests we handle, as the second most significant cost is AWS Lambda, where we run our API. Next up comes S3, which is also unexpected. While our site is hosted on S3, we also use a CloudFront distribution for the content, so we generally expect CloudFront to cost more than S3. We’ll examine these three services in more detail below. Since the API Gateway service makes up most of our monthly cost, we’ll start there.

### API Gateway
Pourwall uses a REST API Gateway to handle incoming requests to our Lambda function. AWS REST API Gateway has many useful features not available in the newer HTTP API Gateway, such as WAF integration and support for canary releases. It does, however, charge more per request. As of writing, REST API Gateways are billed at US$3.50 per million requests, while HTTP API Gateways are billed at US$1.00 per million requests. If you are not using the additional features, switching to an HTTP API Gateway will be cheaper.

If you use a custom domain name for your existing API, the switch to an HTTP API Gateway is done entirely inside of AWS-managed infrastructure. You won’t have to worry about DNS propagation issues or client-side code changes. Set up a new HTTP API Gateway to behave as your existing REST API Gateway, and then update the custom domain mapping in the console when you are ready to switch.

The graphs below illustrate the effect on our cost of switching from a REST to an HTTP gateway. The daily costs are now, on average, 24.8% of what they previously were. The data below covers an interval of 3 months and includes data before and after this change (and all other changes in this blog post).

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/lLhytllhS4ewtnnGmLr0)

Below we can see the number of requests received by both API Gateways during the same period.

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/ai8FBjk2TAGFDwMO0DOg)

### Lambda Cost Reduction
AWS Lambda charges are based on the time your function takes to run. There is also a charge for the number of Lambda invocations made. We will leave this out of our discussion as changes to that require changing client code. You are also charged for the amount of memory configured and ephemeral storage, using the formula below:

> *Lambda Cost =Function Run Time  *(Ephemeral Storage Cost + Memory Cost) *

At a first look, lowering your costs when working with Lambda functions can be done by either optimizing code, lowering its runtime, or by optimizing storage and memory sizes. 

Ephemeral storage is available to the function in the /tmp directory and is between 512MB and 10GB. Unless a Lambda function is used to manipulate large files, it should not need more than the minimum value. As for memory, if the function does not request enough memory, it can fail without logging any errors. The Lambda service will log a message after every run with the amount of memory used. In this case, it will log that all available memory was used. Note that it is also possible to use all available memory without any apparent issues. A Lambda function might also be provisioning too much memory. [Cloudwatch Logs Insights](https://console.aws.amazon.com/cloudwatch/) has a sample query you can run to determine how much memory you are over-provisioning.

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/audqA6eTICPDmTdDhi90)

```
filter @type = "REPORT" 
| stats max(@memorySize / 1024 / 1024) as provisonedMemoryMB, 
  min(@maxMemoryUsed / 1024 / 1024) as smallestMemoryRequestMB,
  avg(@maxMemoryUsed / 1024 / 1024) as avgMemoryUsedMB,
  max(@maxMemoryUsed / 1024 / 1024) as maxMemoryUsedMB,
  provisonedMemoryMB - maxMemoryUsedMB as overProvisionedMB
```

You can run that query selecting your lambda function log group and a sufficiently significant time interval to determine your maximum over-provisioned memory. Usage may vary throughout the day or on different days. Looking at how many log entries you have can help you determine how much time to look over. 
Over-provisioned memory is only an indicator of whether you should change your memory allocation for a Lambda function. This is because Lambda functions are allocated CPU based on the configured memory. When you alter the available memory for the function, you will also alter the function run time. You should check the effect on the function runtime. Be sure you have enough data for comparison. Cloudwatch metrics also contain data on your lambda duration, which can also be found in the Lambda function logs.

The Pourwall API makes some calls to external APIs, affecting the average duration. Since this depends on which users are online at a given time, we take an average of more than 24 hours of data. Below we reproduce the average durations during a single week of July. If we fail to take into account the high-latency calls, we can end up with an average duration that is up to three times as low. This value will also be directly impacted by increases in memory. This might lead us to maintain (or maybe increase) the overprovisioned memory, even though decreasing it will actually lower cost, in our case.

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/n4ZrW34fQb2YMS6ufBZ3)

By using the provided query, we found that around 512 MB of memory was over-provisioned. The table below shows the average durations of our lambda function while varying the configured memory. From the table, we can see that lowering the configured memory by the maximum over-provisioned amount significantly affects the performance, increasing the processing time by 33%. This increase is not enough to affect user experience, and the 50% reduction in cost leads to an overall lower cost. 

![Lambda Duration and Cost for Different Memory Configurations](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/aEgEXOKJTRC7jkbp5PIH)

The graph below shows the average duration after the changes were applied. We can see an increase in the average duration, but this still results in an overall decrease in cost as the amount of memory cut is enough to offset this.

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/mOGbtbJFRN2jEtWxXA5i)

Another factor to consider is the concurrency limit. By default, you can run up to 1000 instances of your Lambda function. You can add concurrency limits to your lambda function to make sure that one function will never take up all the available instances and starve other functions. However, once you set a limit, you may start to see throttling when you reach that limit. This is especially true if the limit is too low. Besides generating errors for the clients, this may also impact your costs. Clients may retry a failed request, incurring further processing costs. 

You can see a considerable decrease in throttling after 08/17, when the concurrency limit was increased.

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/B5IjthWYTOyFVnHRF0jT)

These combined changes resulted in lower monthly costs for our Lambda function, which can be seen in the graph below. Again, there was no significant change in the number of requests being made during this period.

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/bw4Z8tWTQR2nzcq1Zeef)

### S3 Costs

Our service allows users to upload images for backgrounds and other menus. These images are uploaded to an S3 bucket and downloaded by the application website and the TV apps. Our service also allows for the creation of an iframe so users can embed their tap list in another website. The application was using the S3 bucket URLs directly, so all data was being downloaded directly from S3.
Most clients will cache the downloaded images, and response headers will come with a cache-control header. However, client caching is managed by the client application and may not always work. S3 has a logging feature that makes it possible to determine which keys (or files) are being requested. S3 logs are saved to an S3 bucket as multiple CSV files. After joining the data in these files, the below graph was generated. It shows the number of requests for each key over a period of 24 hours. For simplicity, only the top 20 keys were included.

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/ALrxLAMSa6qsLd4CYO70)

The top 20 images are being requested at least 800 times a day, with the top image being requested over 5000 times. Using CloudFront for this content is one way to lower costs. When a cached resource is requested, no request is made to S3, lowering the S3 bandwidth cost. This pattern of repeated requests from the same client will be able to use CloudFront caching.
We set up a CloudFront distribution for the uploaded content and altered our application configurations to use the new CloudFront distribution. Our CloudFront distribution uses Price Class 100, which uses edge locations in North America, Europe, and Israel. This is because most of our users are in North America and Europe, and we do not have a very latency-sensitive application. Using the class will also guarantee that the CloudFront bandwidth will be cheaper than the S3 bandwidth.

We can see the effect of the CloudFront distribution on the S3 costs in the graph below, covering the same 3-month period as before. The valley in cost is due to the free tier offered by AWS. During that period, both the S3 and CloudFront bandwidth falls within this tier, and the account incurs no charges.

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/seIcU2uKRvSiaVLlXXXW)

### Final Analysis
Below, we reproduce the costs for the same 3-month period as above. The month of September is still missing some data, as it had not yet ended at the time of writing. However, we can clearly see the effect of the changes rolled out in the month of August. API Gateway requests are no longer the largest portion of our bill. It is, in fact, now our third most expensive service, behind Lambda and Cloudwatch. 

![](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/xK1NKmFTSA2BBNR8vWfx)

As mentioned in the beginning, we now have new priorities for cost optimization. We should now continue our focus on Lambda optimizations but also turn an eye to Cloudwatch. In a future blog post, we will dive deeper into how we lowered Lambda costs by getting rid of polling through the use of WebSockets.

Optimizing your AWS Serverless Costs

### Overview
Building apps that function well across multiple platforms has been challenging at times, but now there are tools to speed up the process. Flutter is a great option for creating cross-platform apps for iOS, Android, Web, Windows, and macOS. With one codebase you can create a native app for each platform with very little custom code.

Out of the box, the app will look and behave similarly, though not identically, on each platform because Flutter automatically accounts for platform differences via platform adaptations.

> Things that are behaviors of the OS environment (such as text editing and scrolling) and that would be ‘wrong’ if a different behavior took place.
Flutter Docs – [Platform Adaptions](https://docs.flutter.dev/resources/platform-adaptations)

For example, here’s a standard text field in both iOS and Android. The form fields look nearly identical, but the text selection interface is customized per platform.

![Android text selection](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/kuBIOL8kSWGoXI2zfVdU)

![iOS text selection](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/mUk0TCsATk60qRT2EFHg)

### Platform Aware
In many projects, you’ll want iOS users to see the more familiar iOS design system. To do this, you’ll want to use platform aware widgets. These widgets are aware of what platform they’re being displayed on and will render the UI that’s appropriate.  Let’s take a look at the three ways you can put platform aware widgets to work in Flutter.

### Flutter’s Built-in Adaptive Widgets
Flutter has some built-in ‘adaptive’ widgets which automatically render appropriately for iOS and Android. For example, the standard Icon widget renders the same icon for both platforms.

```
Icon(Icons.share);
```

![standard Flutter icons](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/24vavxbpRwiCAtMXN7ss)

But if you use the adaptive version, it will render the appropriate version for each OS.

```
Icon(Icons.adaptive.share);
```

![adaptive Flutter icons](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/XK9UCLYDTEeXuKSikGaL)

### Roll Your Own Adaptive Widgets
If Flutter doesn’t offer the adaptive widget you need, you can create your own pretty easily. Check the OS via the Platform class and return the appropriate widget.

```
 _makeSlider() {
      if (Platform.isAndroid) {
        return Slider(
          min: 0,
          max: 10,
          value: 1,
          onChanged: (val) => print(val),
        );
      } else if (Platform.isIOS) {
        return CupertinoSlider(
          min: 0,
          max: 10,
          value: 1,
          onChanged: (val) => print(val),
        );
      }
    }
```

![Material and Cupertino version of a Slider](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/n5fjEiDQTGIhPXvFAlUS)

While rolling your own adaptive widgets does work,  it has a downside. If you go this route, your code will be littered with these platform checks and be difficult to maintain.

### *flutter_platform_widgets*
Luckily, there’s a package available that solves this problem. [flutter_platform_widgets](https://pub.dev/packages/flutter_platform_widgets) offers 23 platform aware widgets and has done the work to make sure they integrate well together.

To get started just call

```flutter pub add flutter_platform_widgets```

Here is the code for both the platform aware and unaware components.

```
import 'package:flutter/cupertino.dart';
import 'package:flutter/material.dart';
import 'package:flutter_platform_widgets/flutter_platform_widgets.dart';
import 'package:platformaware/screens/aware_home_screen.dart';

class AwareApp extends StatelessWidget {
  const AwareApp({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return const PlatformApp(
      localizationsDelegates: [
        DefaultMaterialLocalizations.delegate,
        DefaultWidgetsLocalizations.delegate,
        DefaultCupertinoLocalizations.delegate,
      ],
      home: AwareHomeScreen(),
    );
  }
}
```
[aware_app.dart](https://gist.github.com/adamontherun/cda3afcf74308f5219671c14846b1a5d#file-aware_app-dart)
```
import 'package:flutter/material.dart';
import 'package:flutter_platform_widgets/flutter_platform_widgets.dart';
import 'package:platformaware/screens/dummy_screen.dart';

class AwareHomeScreen extends StatelessWidget {
  const AwareHomeScreen({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return PlatformScaffold(
      appBar:
          PlatformAppBar(title: const Text('Platform Demo'), trailingActions: [
        PlatformIconButton(
            onPressed: () {
              Navigator.push(
                  context,
                  platformPageRoute(
                    context: context,
                    builder: (_) => const DummyScreen(),
                  ));
            },
            icon: const Icon(Icons.add))
      ]),
      body: Padding(
        padding: const EdgeInsets.all(20.0),
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          crossAxisAlignment: CrossAxisAlignment.center,
          children: [
            PlatformTextField(
              hintText: 'First name',
            ),
            PlatformSwitch(value: true, onChanged: (value) => print('update')),
            PlatformSlider(
              min: 0,
              max: 10,
              value: 1,
              onChanged: (val) => print(val),
            ),
            PlatformTextButton(
                onPressed: () {
                  showPlatformDatePicker(
                    context: context,
                    initialDate: DateTime.now(),
                    firstDate:
                        DateTime.now().subtract(const Duration(days: 1000)),
                    lastDate: DateTime.now().add(const Duration(days: 1000)),
                  );
                },
                child: const Text('choose date')),
            PlatformTextButton(
              onPressed: (() {
                showPlatformDialog(
                  context: context,
                  builder: (_) => PlatformAlertDialog(
                    title: const Text('Alert'),
                    content: const Text('Some content'),
                    actions: <Widget>[
                      PlatformDialogAction(
                        child: PlatformText('Enable'),
                        onPressed: () => Navigator.pop(context),
                      ),
                      PlatformDialogAction(
                        child: PlatformText('Disable'),
                        onPressed: () => Navigator.pop(context),
                      ),
                    ],
                  ),
                );
              }),
              child: const Text('show a dialog'),
            ),
            PlatformPopupMenu(
                options: [
                  PopupMenuOption(
                    label: 'Option One',
                  ),
                  PopupMenuOption(
                    label: 'Option Two',
                  ),
                  PopupMenuOption(
                    label: 'Option Three',
                  )
                ],
                icon: const Icon(
                  Icons.more_horiz,
                )),
          ],
        ),
      ),
    );
  }
}
```
[aware_home_screen.dart](https://gist.github.com/adamontherun/cda3afcf74308f5219671c14846b1a5d#file-aware_home_screen-dart)
```
import 'package:flutter/cupertino.dart';
import 'package:flutter/material.dart';
import 'package:platformaware/screens/unaware_home_screen.dart';

class UnawareApp extends StatelessWidget {
  const UnawareApp({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return const MaterialApp(
      localizationsDelegates: [
        DefaultMaterialLocalizations.delegate,
        DefaultWidgetsLocalizations.delegate,
        DefaultCupertinoLocalizations.delegate,
      ],
      home: UnawareHomeScreen(),
    );
  }
}
```
[unaware_app.dart](https://gist.github.com/adamontherun/cda3afcf74308f5219671c14846b1a5d#file-unaware_app-dart)
```
import 'package:flutter/material.dart';
import 'package:platformaware/screens/dummy_screen.dart';

class UnawareHomeScreen extends StatelessWidget {
  const UnawareHomeScreen({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(title: const Text('Platform Demo'), actions: [
        IconButton(
            onPressed: () {
              Navigator.push(
                context,
                MaterialPageRoute(builder: (context) => const DummyScreen()),
              );
            },
            icon: const Icon(Icons.add))
      ]),
      body: Padding(
        padding: const EdgeInsets.all(20.0),
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          crossAxisAlignment: CrossAxisAlignment.center,
          children: [
            const TextField(
              decoration: InputDecoration(
                hintText: 'First name',
              ),
            ),
            Switch(value: true, onChanged: (value) => print('update')),
            Slider(
              min: 0,
              max: 10,
              value: 1,
              onChanged: (val) => print(val),
            ),
            ElevatedButton(
                onPressed: () {
                  showDatePicker(
                    context: context,
                    initialDate: DateTime.now(),
                    firstDate:
                        DateTime.now().subtract(const Duration(days: 1000)),
                    lastDate: DateTime.now().add(const Duration(days: 1000)),
                  );
                },
                child: const Text('choose date')),
            ElevatedButton(
                onPressed: () {
                  showDialog(
                      context: context,
                      builder: ((context) {
                        return AlertDialog(
                          title: const Text('alert!'),
                          content: const Text('some alert content'),
                          actions: [
                            TextButton(
                              child: const Text('Disable'),
                              onPressed: () {
                                Navigator.of(context).pop();
                              },
                            ),
                            TextButton(
                              child: const Text('Enable'),
                              onPressed: () {
                                Navigator.of(context).pop();
                              },
                            ),
                          ],
                        );
                      }));
                },
                child: const Text('show an alert')),
            PopupMenuButton(
              itemBuilder: (context) {
                return const [
                  PopupMenuItem(
                    child: Text('one'),
                  ),
                  PopupMenuItem(
                    child: Text('two'),
                  ),
                  PopupMenuItem(
                    child: Text('three'),
                  ),
                ];
              },
            ),
          ],
        ),
      ),
    );
  }
}
```
[unaware_home_screen.dart](https://gist.github.com/adamontherun/cda3afcf74308f5219671c14846b1a5d#file-unaware_home_screen-dart)

Let’s take a look at the difference when you use a platform aware widget. Here’s a  version that’s not platform aware. Notice there are some minor differences, but overall its similar though  looks a little ‘off’ for iOS.

![video: Standard platform unaware version](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/B3yQivcQfeNSH0t846Qw)

Now let’s look at  the app updated to use the *flutter_platform_widgets* components with no other changes. Notice that the Android version looks the same, but the iOS version looks like a typical iOS app.

![video: flutter_platform_widgets platform aware version](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/uAF3mbsSSsm4T99yD5ag)

The *flutter_platform_widgets* package offers the ability to both extend and theme all of these components. With just a little extra work, your Flutter app can look at home on both iOS and Android, creating a seamless experience for every user.

Your Flutter app can look at home on both iOS and Android, creating a seamless experience for every user, by leveraging platform aware widgets.

Flutter Platform Aware Widgets

Best-in class software innovates and evolves to keep customers engaged. One way to improve and add features without disrupting the current user experience is to deploy a canary release.

Canary releases get their name from an old coal mining practice of taking canaries into the mine to test for deadly carbon monoxide. Carbon monoxide is odorless, and would affect a small bird much faster than a miner. So if you were deep in a mine and your canary stopped singing and became sluggish, you knew it was time to get out quickly. Here in the coding world, we can use a similar warning system to test new features and code changes.

## Here’s how it works
When you want to introduce a new feature or code change, you run two different versions of your code in parallel. Most of your users will still be accessing the current version of your site or service – think of them as your miners. Then, a small percentage of users – the canaries – will experience the new version. This allows you to test your new features or changes with real world users while limiting the potential impact of any bugs in the new release.

Once you’re confident that the new version is working as it should, you can then increase the percentage of users using the new version of your code. This slow rollout will allow you to diagnose any scaling issues that might not initially show up with your small test group.

## AWS Cloudfront 
Cloudfront is a Content Delivery Network (CDN) provided by AWS. It helps developers make their sites scale better and with lower latencies by caching site content locally across multiple regions of the world, while only maintaining a single instance of your site. It’s a great tool for developers and is widely used, but AWS Cloudfront doesn’t have native support for canary releases. Fortunately, there’s a workaround: it does have support for running custom code on all requests through Cloudfront Functions and lambda@edge. So let’s talk about how to leverage lambda@edge functions to implement a canary release.

## lambda@edge
The lambda@edge AWS service allows developers to deploy lambda functions to all Cloudfront regions, so they can be called by Cloudfront with low latencies. lambda@edge functions have some different limitations than regular lambda functions, and you can check those out [here](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/edge-functions-restrictions.html).

When you associate a lambda@edge function with a Cloudfront distribution, the function version that is associated will be deployed to all regions. This allows for lower latency, but be aware that the logs for the lambda function will be created in the region where the request was handled.

## A Canary Release on Cloudfront
Now, let’s talk details. Hosting a SPA on S3 using Cloudfront as a cache is a simple way to host your site. You can use the static site hosting option from S3 for a public site, or you can keep your S3 bucket private and access the content through Cloudfront using an OAI. If you plan to use an OAI, you’ll probably need to do some editing to the requests with a lambda@edge or a Cloudfront function to handle paths correctly. 

![Untitled.png](https://us-east-1.graphassets.com/A4z9AMx4fRRelBcpGyU8Nz/0UnENPkETv2kQdKHKV0i)

To  run a canary release, just add a lambda@edge function to handle the requests that cloudfront will be making to the origin, in this case the S3 bucket. The code below will change the origin based on the value of the header bucket. This header should be the header used in the cache behavior in Cloudfront. This will reduce the number of times the lambda functions need to be called.

```
"use strict";
exports.handler = (event, context, callback) => {
  const request = event.records[0].cf.request;
  if (request.headers.cookie) {
    request.headers.cookie.foreach(function (cookie) {
      if (cookie.value.startswith("mycookiename=")) {
        const domainname = "new-release-bucket.s3.amazonaws.com";
        request.origin.s3.domainname = domainname;
        request.headers["host"] = [{ key: "host", value: domainname }];
      }
    });
  }
  callback(null, request);
};
```

It’s best practice to set a cookie before making the request, but you can also make use of another function attached to the viewer request or response event to set the cookie. Using a cookie will also keep the user on the new release throughout all the requests made to the site.

## Where can you grow from here?
The great thing about deploying canary releases is that you can dream big and keep risks small. What are some of the features and add-ons your customer’s have been asking for? What are the best-in-class apps doing that you’d like to be doing? What new feature would make your software stand out? Pick one, start coding, and let the canaries lead the way! 

Innovation without Disruption

The Canary Release

Articles

Receive our quarterly update "The Buzz" delivering industry insights and updates from the Beta Acid crew.
