IMAGE: A photo of 0x41434f after his shift at the fulfillment center.
I've always noticed that founders tend to build solutions for problems they're directly exposed to. It's why we're seeing a surge in AI-powered SaaS applications targeting coding, sales, marketing, and customer service. Founders spot inefficiencies in their own workflows and set out to automate them 1.
Image Credits: a16z
But my path has been a bit different.
Growing up, I worked alongside my parents in their businesses. This experience exposed me to the fragmented and offline nature of the industrial and manufacturing sectors—industries often overlooked and considered "unsexy" in the tech world. Yet, I've always believed there's immense untapped potential there.
Earlier this year, between March and July, I decided to immerse myself in a completely different industry to gain new perspectives. After receiving incorrect orders from a meal kit service not once, but four times, my curiosity was piqued. I wanted to understand why these companies, which rely heavily on logistics and precision, were making such costly mistakes.
So, I took a night shift job at a local meal kit fulfillment center. Friends thought I was crazy—why take on a night job when I didn't need to or even take a fulfillment center job in the first place? But I was on a mission to uncover the root causes of these recurring errors.
Upon being hired and during orientation, I discovered something astonishing: the company was losing nearly a million dollars due to packing errors. For every incorrect order, customers received credits, and the food couldn't be returned—essentially becoming freebies. Even the training director admitted he'd personally received several wrong orders multiple times.
Working there, I observed firsthand how small lapses in the packing process led to significant errors. It was both eye-opening and frustrating. I couldn't help but think about how Generative Physical AI 2 and computer vision 3 could be harnessed to improve quality assurance, ensuring every box was packed correctly.
Browsing online forums, I found that I wasn't alone in noticing these issues. Many customers on Reddit shared their frustrations about receiving incorrect meals and missing ingredients from meal kit services 4. For example, users discussed how receiving wrong meals and missing ingredients was throwing off their meal planning and causing significant inconvenience 5. It was clear that this was a widespread problem affecting both customer satisfaction and company profitability.
What truly validated my approach was stumbling upon discussions where industry leaders emphasized the importance of gaining firsthand experience before innovating in a field. In a post on r/ycombinator titled "Why are you working in a space with 0 domain expertise?", the author questioned why so many founders choose to work in fields where they have no experience. They suggested that if you don't have experience in an area, you should use your personal interests to guide you and gain experience in that field—even if it means working undercover for a while. This approach, they argued, could provide invaluable insights and make for a compelling story to share with future customers, investors, and the press.
Image Credits: Reddit
Around the same time, Garry Tan, the CEO of Y Combinator, shared a thought that resonated deeply with me. He mentioned that "going undercover" by working a non-technical job you're interested in and where you think large language models (LLMs) could automate tasks is becoming an increasingly common way to gain meaningful domain expertise quickly.
tbh this is a banger post
“Going undercover” working a nontechnical job that you have an interest in and that you think LLMs could automate is a pretty wild and increasingly common way to get meaningful domain expertise fast
pic.twitter.com/qH3A7HMYIu
— Garry Tan (@garrytan) August 24, 2024
Seeing these discussions emerge after my experience from March to July was both affirming and exciting. It felt gratifying to know that I was already thinking along these lines before these ideas became more prevalent in the startup community. It was an indication that I was on the right track, thinking ahead of the curve.
This journey reinforced my belief that real innovation often comes from stepping outside familiar territories. By immersing myself in an industry unrelated to my background, I uncovered problems that weren't immediately apparent from the outside.
Whether it's applying AI to reduce errors in meal kit deliveries or bringing digital solutions to the industrial sectors I grew up around, the key is to dive deep, observe, and understand the challenges people face every day.
It's not always about chasing the latest trend. Sometimes, the most impactful opportunities lie in the overlooked corners of the world, waiting for someone to pay attention and think ahead.
Inside the Meal Kit Fulfillment Operation
Working the third shift from 10:00 pm to 6:30 am, Thursday through Monday, I immersed myself in the fast-paced world of meal kit production. Each night began with a stand-up meeting where supervisors shared our goals: assembling around 25,000 meal bags and 5,000 portioning per night on Thursday and Friday, and shipping about 4,000 to 5,000 from Saturday till Monday, depending on attendance.
The operation was a complex orchestra involving several key roles:
- Production Associates: Responsible for labeling, portioning, meal bagging/kitting, and shipping orders.
- Gatekeepers: Managed inventory submissions and tagging, ensuring accurate counts and organization.
- Warehouse Dropzone Gatekeepers: Handled requests for bulk products and packaging, coordinating closely with production and warehouse teams.
- Supervisors: Executed the production plan, coordinated setups, and managed teams of leads and associates.
- Food Safety and Quality Assurance (FSQA): Ensured the accuracy and quality of all products, performing checks throughout the production process.
Despite the structured workflow and clearly defined roles, I noticed several recurring problems that led to errors.
The Challenges We Faced
1. Communication Breakdowns
Supervisors sometimes weren't sure how many shipping lines to run or how many bags we needed to pack or portion. They occasionally gave confusing information to team leads. With three shifts running around the clock and handovers between them, miscommunication was common. The next shift often inherited whatever was left on the shipping line or dropzone, leading to inconsistencies and confusion.
2. Errors in Meal Bagging
Production Associates, like myself, occasionally didn't put the correct items in meal bags or missed certain ingredients entirely. For example, if a recipe called for two packets of an item, we might include only one. There wasn't always a clear guideline on whether to discard questionable products, so sometimes subpar items made it into bags. The pressure to meet high output targets often exacerbated these mistakes.
3. Mistakes on the Shipping Line
On the shipping lines, associates handle not only proteins but also complete meal bags, recipe cards, and other items needed to fulfill customer orders, ensuring that each box was accurately assembled based on specific orders. Despite this comprehensive responsibility, errors predominantly arise from meal bagging and proteins. Associates sometimes placed the wrong protein in boxes, miscounted items, or omitted proteins altogether. This often happened because we were moving quickly or assumed that the protein layout remained the same from week to week. However, the protein shelves changed frequently, leading to confusion. Associates who had worked previous shifts might mistakenly grab items based on memory rather than checking the current setup.
4. Warehouse and Dropzone Mix-ups
Warehouse and Dropzone staff sometimes brought out incorrect ingredients, proteins, or produce. Runners responsible for replenishing the shipping lines and meal bagging tables occasionally supplied the wrong items because they looked similar or didn't thoroughly check the ingredient IDs, names, or sizes. I worked as a runner/replenisher and saw firsthand how easy it was to make these mistakes. Sometimes, items from the previous week were mistakenly used, or products had the wrong ounce size. Similar packaging and naming conventions added to the confusion.
5. Inaccurate Tagging and Setup Errors
Gatekeepers might mislabel stacks of meal bags, and Team Leads sometimes set up tables with the wrong ingredients or produce. This includes incorrectly setting up the shipping line with wrong proteins, meals, and already packed meal bags labeled E1 to E33. With over 32 types of proteins varying in ounces and cuts (e.g., 8, 10, 20 oz; cutlet vs diced vs breast), as well as diverse meal options like Southwest-Style Chicken Enchilada Bowl, the complexity increases. For example, a correct box should contain:
- Ice
- Recipe card
- 20 oz. Diced Boneless Skinless Chicken Breasts
- 8 oz. Black Beans
- 7½ oz. Minute Rice
- 5 fl. oz. Enchilada Sauce
- 5 oz. Corn Kernels
- 3 oz. Sour Cream
- 3 oz. Shredded Cheddar Cheese
- 1 oz. Tortilla Strips
- 4 tsp. Chicken Broth Concentrate
- 3 tsp. Fajita Seasoning
Associates would then unknowingly use incorrect items for shipping, perpetuating these errors by inheriting mislabeled or incorrectly set up items from previous shifts.
Personal Reflections on Workplace Culture
Experiencing the pervasive culture of fear and favoritism at the meal kit fulfillment center had a profound impact on me. On one hand, I was disheartened by how some employees were treated without dignity. On the other hand, I observed behaviors that undermined productivity and team cohesion, such as employees taking excessive bathroom breaks, engaging in unproductive conversations throughout the night, and manipulating time records to steal hours. These actions, while frustrating, did not justify the toxic environment fostered by management.
Coming from a background as a software security engineer, I had never encountered such a hostile workplace culture. This stark contrast reinforced my belief in the importance of respectful and transparent leadership. As a founder, I draw immense inspiration from my parents, Aunt, Professor D, Ms. L, and Mr. T. I often remind myself that failing to lead with integrity and compassion would be a disgrace to these role models. Their guidance shapes my approach to building a company culture that values dignity, fairness, and mutual respect.
Beyond the broader cultural issues, there were personal interactions that stood out. I was known for being diligent and striving for accuracy, which made me memorable to my colleagues. Some noticed that I initially wore glasses but stopped wearing them after the first month. This change sparked curiosity among the team members. The toughest girl on the shipping line, who initially seemed intimidating, turned out to be the sweetest. She always teased me about my glasses and made an effort to joke around, creating a friendly rapport. These small interactions humanized the workplace and showed that beneath the surface tensions, there were genuine connections being formed.
The Impact of These Errors
These mistakes weren't just operational hiccups—they had real consequences:
- Customer Dissatisfaction: Receiving incorrect or incomplete orders frustrated customers and eroded trust in the brand.
- Financial Losses: The company was losing nearly a million dollars due to these errors. Incorrect orders couldn't be returned, and customers received credits as compensation.
- Employee Morale: Constant errors led to a stressful work environment. Associates were often rushing, increasing the likelihood of mistakes and creating a vicious cycle.
Lessons Learned and Shaping My Leadership Philosophy
My undercover experience at the fulfillment center was more than just an operational deep dive; it was a pivotal moment that reshaped my understanding of effective leadership and team dynamics. Witnessing firsthand the detrimental effects of fear-based management and favoritism highlighted the critical need for transparency and equity within any organization.
These insights have directly influenced my approach as a founder. I prioritize creating an inclusive environment where every team member feels valued and empowered to contribute their best. By fostering open communication and recognizing individual efforts without bias, I aim to build a resilient and motivated team. Additionally, integrating AI and computer vision solutions into our workflows is not just about enhancing efficiency but also about alleviating the pressure on employees, allowing them to focus on meaningful tasks rather than being bogged down by repetitive errors.
Moreover, the challenges I faced underscored the importance of adaptability and proactive problem-solving. Implementing AI-driven quality assurance tools, for instance, stems from my commitment to reducing human error and improving operational transparency. This ensures that our solutions are not only technologically advanced but also human-centric, addressing real pain points with empathy and innovation.
The personal connections I made, like the friendly teasing from the toughest girl on the shipping line, taught me the value of building strong, supportive relationships within a team. These relationships foster a positive work environment where employees feel comfortable and motivated to perform their best.
Roles Designed to Prevent Errors
Interestingly, the job descriptions for each role were specifically designed to prevent these issues:
- Production Associates were expected to have strong attention to detail, use proper measuring utensils, and ensure accuracy in picking ingredients.
- Gatekeepers were responsible for accurate inventory submissions and proper tagging to maintain organization and traceability.
- Dropzone Gatekeepers were tasked with verifying information before production consumption, ensuring the correct items were brought to the floor.
- Supervisors were meant to coordinate effectively, provide clear instructions, and ensure that standard operating procedures were followed.
- FSQA teams were in place to conduct on-the-line checks and ensure products met quality standards.
Despite these safeguards, the human element—miscommunication, fatigue, and assumptions—often led to errors slipping through the cracks.
Envisioning an AI-Powered Solution Focused on Quality Assurance
Witnessing these challenges firsthand, I saw an opportunity for generative physical artificial intelligence and computer vision to enhance quality assurance. Here's how:
1. Real-Time Ingredient Verification
Implement computer vision systems at meal bagging and portioning stations. Cameras equipped with AI could:
- Scan Items: Verify that the correct ingredients are being placed into meal bags.
- Count Quantities: Ensure the correct number of each item is included (e.g., two packets instead of one).
- Alert for Errors: Provide immediate feedback to associates if an incorrect item is detected.
2. Dynamic Shipping Line Monitoring
On the shipping lines, AI could:
- Track Protein Placement: Use sensors and cameras to monitor which proteins are being placed into boxes, ensuring they match the customer's order.
- Adjust for Shelf Changes: Update associates in real-time about changes in protein shelf arrangements, reducing confusion from week to week.
- Prevent Omissions: Alert associates if an item is missed before the box is sealed.
3. Enhanced Communication Tools for Supervisors
Develop an AI-driven production planning system that:
- Standardizes Shift Handover: Provides detailed reports on the current state of the production floor, including what's on the shipping line or dropzone.
- Predicts Needs: Analyzes data to recommend how many shipping lines to run and how many bags to pack or portion based on real-time demand.
- Reduces Miscommunication: Offers a centralized platform where supervisors, team leads, and associates can access up-to-date information.
4. Intelligent Inventory and Tagging Systems
Implement smart tagging with RFID technology:
- Accurate Tagging: Gatekeepers can use RFID tags that automatically update inventory systems when scanned as opposed to the paper labels.
- Track Item Movement: Monitor the flow of ingredients from the warehouse to production to shipping.
- Prevent Use of Incorrect Items: The system can flag items from previous weeks or with incorrect sizes before they reach the production floor.
5. Augmented Reality (AR) for Training and Guidance
Use AR headsets or tablets to assist associates:
- Visual Cues: Overlays can show associates exactly where items should go, what ingredients to pick, and quantities required.
- Interactive Training: New associates can receive on-the-spot training modules, reducing onboarding time and errors.
- Updates and Alerts: Instant notifications about changes in processes or product placements.
6. AI-Powered Quality Assurance Checks (currently building this)
Enhance the role of FSQA teams with AI tools:
- Automated Inspections: Use computer vision to perform initial quality checks on products, flagging any deviations from standards.
- Data Analytics: Collect data on error patterns to identify root causes and address them proactively.
- Support for Human Inspectors: Allow FSQA staff to focus on complex issues that require human judgment while AI handles routine checks.
Expanded Future Vision
The potential of physical AI and computer vision doesn't stop at meal kit fulfillment centers. Leading companies like Sam’s Club and Amazon are already leveraging these technologies to revolutionize their operations.
Sam’s Club AI-Powered Exit Technology
Sam’s Club has recently unveiled an AI-driven exit technology designed to streamline the checkout process at my local club 6. Instead of requiring store staff to manually check members’ purchases against their receipts, Sam’s Club uses a combination of computer vision and digital technology to capture images of customers’ carts and verify payment.
IMAGE: A photo of 0x41434f grocery shopping at a local Sam's Club with exit cameras.
The system works via a combination of computer vision and digital tech that captures images of customers’ carts and then verifies payment for the items in their shopping cart. Sam’s Club says AI is used in the background to speed up the process. The AI also learns and improves over time as thousands of exit transactions across locations are analyzed.
Before the technology was put into place, Sam’s Club members had to queue up at the store’s exit to wait to have their receipts checked. The new solution keeps them moving along and frees up store staff to focus on other tasks.
Amazon’s Vision-Assisted Package Retrieval (VAPR)
Amazon is another pioneer in integrating AI with logistics. Their Vision-Assisted Package Retrieval (VAPR) technology enables delivery drivers to quickly identify and retrieve packages using computer vision 7. By projecting visual cues onto packages, VAPR allows drivers to grab the correct items without manually checking each one.
These real-world applications demonstrate the transformative power of AI and computer vision in enhancing operational efficiency, reducing errors, and improving customer satisfaction across various industries.
Bridging Technology and Human Effort
The goal isn't to replace human workers but to empower them. By integrating AI and computer vision into the workflow, we can:
- Reduce Errors: Automated checks catch mistakes before they reach the customer.
- Enhance Efficiency: Associates can focus on their tasks without constant worry about making errors.
- Improve Morale: A smoother operation reduces stress and increases job satisfaction.
Conclusion
My undercover experience at the meal kit fulfillment center provided invaluable insights into the operational challenges and workplace dynamics that drive errors and inefficiencies. By immersing myself in this environment, I not only identified critical areas where AI and computer vision can make a tangible difference but also reinforced my commitment to fostering a respectful and transparent company culture.
Integrating advanced technologies with a human-centric approach ensures that we address real-world problems while empowering employees to perform their best. Whether it's reducing packing errors in meal kit deliveries or revolutionizing the industrial sectors I grew up around, the key lies in understanding the intricacies of the workflow and applying innovative solutions that enhance both efficiency and employee satisfaction.
It's not always about chasing the latest trend. Sometimes, the most impactful opportunities lie in the overlooked corners of the world, waiting for someone to pay attention and think ahead.
To my fellow Technologists
I encourage fellow founders and technologists to seek out unconventional industries and immerse themselves in environments where they can uncover hidden challenges. Consider taking on roles that may seem unrelated to your field of expertise—these experiences can provide unique insights and inspire innovative solutions.
Moreover, sharing your experiences and learning from others can foster a community of problem solvers dedicated to making meaningful improvements across all sectors.
To those inspired by my journey, I invite you to share your own stories or contemplate similar undercover roles. Together, we can drive innovation in the most unexpected places and create impactful solutions that benefit both businesses and customers.
I am building an open-source AI-Powered Quality Assurance Checks project based on my experience as a Product Associate at the Meal Kit Fulfillment Center.
Project FulfillmentChecker Overview
Purpose: To create an app that enhance the role of FSQA teams with AI tools.
Goals:
- Automated Inspections: Use computer vision to perform initial quality checks on products, flagging any deviations from standards.
- Data Analytics: Collect data on error patterns to identify root causes and address them proactively.
- Support for Human Inspectors: Allow FSQA staff to focus on complex issues that require human judgment while AI handles routine checks.
Key Features
Technological Stack
- Raspberry Pi 4 Model B
- Raspberry Pi AI Camera
- Create custom neural network model with TensorFlow or PyTorch.
Challenges
- Need about 50 - 100 images per class of mealbags, proteins, etc, so I have been buying meals and taking pictures to easily label, train, and deploy the AI model.