A Collaborative Partnership in Cloud Computing Hardware 

In the ever-evolving computing industry, collaborative partnerships that foster innovation are essential for pushing the boundaries of technological capabilities. One such collaboration unfolded between Oxide Computer Company (Oxide) and Benchmark. Oxide is an innovative technology company serving some of the U.S. National Laboratories (the leading institutions for scientific innovation in the United States) and providing businesses with servers that harness the power of the cloud while maintaining hosting control and security on premises. Oxide’s desire to design and build industry-disruptive server racks led them to Benchmark. The unique partnership allowed Oxide to leverage Benchmark’s expertise in designing systems with large application-specific integrated circuits (ASICs) and high-speed computers. The following article is an interview with Amy Hocraffer, Design Engineering Technical Product Manager at Benchmark, and Erik Anderson, Operations Manager at Oxide, focusing on this successful partnership.

Can you share the origins of this collaborative partnership? What made Benchmark and Oxide decide to work together on this project?

Erik:

We started evaluating contract manufacturers in early 2020. As with other key partners, we were looking foremost for strong value alignment (see our principles), followed by technical expertise and strong business fundamentals. It was clear early on that Benchmark understood our product vision, shared our philosophy, and saw the same opportunity in the market. Benchmark’s industry experience, engineering competence, and manufacturing prowess allowed us to move quickly through several design iterations and successfully bring our product to market.

Amy:

As Erik mentioned, we started our discussions in early 2020 during the peak of COVID, so visits and face-to-face meetings weren’t available to us. We had preliminary discussions focused on ensuring our two organizations shared key values and ways of working. The Benchmark team worked to understand Oxide’s needs and both Oxide and Benchmark technical leaders worked together to understand the immediate and long-term project needs.

Once we kicked off the project, we conducted focused feasibility studies to confirm some of the highest risk (and most critical planned) design elements. Through those studies, our teams continued to develop a shared mode of working and confirmed our joint commitment to partnership and quality prior to starting the larger development effort that led to the full rack system.

The partnership required collaboration across different areas of expertise. Could you elaborate on how Benchmark’s mechanical engineering team and Oxide’s software and electrical engineering teams came together to address challenges?

Erik:

Oxide is largely a remote company. Approximately two-thirds of our staff doesn’t normally work from our office in Emeryville, California. We had our own doubts about bringing an ambitious product like ours to market with a distributed team—but we did it. We put a lot of thought into how to organize our company to build trust and communicate with a team that spans twenty one time zones. As it turns out, that had spillover benefits for how we collaborate externally. As an example, we use a written request for discussion (RFD) process to explore and document determinations for product-related design topics as well as business processes.

Our RFDs complemented the engineering statement of work to drive specific feasibility studies and detailed design work. To keep track of it all, we used GitHub to track design issues, shared files with the Google Workspace suite, and created a channel on Benchmark’s Microsoft Teams for chat. We held recurring and ad hoc meetings on various topics and recorded them. Those recordings were useful for later reference and helped with asynchronous communication.

There are times, however, when in-person collaboration is the only way to go: Engineering bring-up on early prototypes, working through a crisis, and the run-up to the first customer shipment. During those times, I was always grateful to be building at Benchmark in Minnesota so we didn’t have to fly halfway across the globe.

Amy:

The RFD process facilitated in-depth discussions and collaboration as we defined the rack system and its components, helping Oxide, Benchmark, and our distributed teams stay aligned and ahead of issues. Our many modes of shared communications kept our organizations working as one team, even when COVID canceled travel and forced people to work from home. We were able to use video calls and chats to bridge that gap with real-time demonstration until we were able to bring the Oxide team on-site for key milestones, such as the Engineering Validation Test (EVT) builds and initial bring-up.

Can you describe how Benchmark’s mechanical engineering and industry expertise contributed to the success of the Oxide project?

Erik:

Working with Benchmark’s mechanical engineering (ME) team was a highlight. The ME team demonstrated strong technical leadership and customer focus every step of the way from rack frame and enclosure design, tolerance stack-up analyses, and thermal simulations to supplier selection and rapid prototyping. Benchmark’s deep expertise in building large-scale computers gave us a better understanding of design tradeoffs as we sought to balance product differentiation with cost, reliability, and serviceability down the road.

Amy:

Benchmark’s experience in high-speed compute and complex systems helped us to foresee potential pitfalls in the design early in the process and to know what elements might need extra tolerance analysis or prototyping to ensure a robust design that would meet Oxide’s needs. We were able to recommend typical industry standards and compliance elements early in the project so Oxide could consider which would apply to their rack before the design was too mature, minimizing re-design and churn.

Given the complexities of working with sheet metal and the supply chain disruptions caused by the recent pandemic, how did both companies manage to maintain effective communication and collaboration during this critical time?

Erik:

Getting through all the supply chain disruptions caused by the COVID-19 pandemic was tough. As a startup designing multiple subsystems of our product simultaneously, we had frequent changes to our bill of materials (BOM). We decided to procure and consign essentially all components due to supply or allocation uncertainty and a desire to understand our product cost structure at a very granular level. In hindsight, we probably should have taken Benchmark up on their offer to handle procurement for us, certainly for the lower cost items.

We created several time consuming and costly headaches for ourselves by operating outside Benchmark’s normal procurement and inventory management processes. Given Benchmark’s large purchasing volume with many suppliers and distributors—and therefore preferential pricing—it’s not at all clear that we saved on cost by purchasing components ourselves. Despite our decision to handle procurement outside of their normal processes, Benchmark helped us in good faith with receiving and handling the inventory. Benchmark was also instrumental in helping us select suppliers for certain categories, particularly sheet metal parts.

Amy:

Benchmark has long-term relationships with several key suppliers that helped us to advise Oxide on lead times, potential fabrication challenges, and vendors with the necessary expertise in some of the trickier areas of the design. Oxide also encouraged buy-aheads as needed which allowed us to reduce lead time even during the worst of the pandemic-driven disruptions. Finally, where possible, Benchmark made use of 3D printing and other rapid prototyping methods to allow us to get hands-on with components to check form, fit, and function while waiting for the real part.

What strategies were employed (and continue to be employed) to foster transparent communication and trust between the two companies throughout the project’s lifecycle?

Erik:

The lifecycle transition from the development to production phase is a tricky one, especially for a startup. Benchmark’s organizational structure has different teams for each phase but has a framework and set of best practices for managing the transition. We worked with Benchmark to build our final batch of development hardware (design validation test, or DVT) with the production side of the house. In retrospect, that was a good decision because it forced Oxide to shore up internal processes like engineering documentation control and test infrastructure. It also allowed us to start building relationships with that side of Benchmark.

Regardless of product lifecycle phase, the keys to success have been the same: trust and communication. Benchmark worked through their strict internal security processes to grant Oxide engineers access to their facility for hardware bring-up events and ongoing access for test and process development. Given our small company size, we punch above our weight for Benchmark’s attention with weekly on-site presence and engagement and we appreciate Benchmark for graciously extending that level of access. Even though we’ve now moved beyond development into production, we have daily meetings to review production status, weekly meetings to review inventory, and a quarterly business review for executive-level updates.

Amy:

Throughout the project (and today), we used frequent and detailed touchpoints with the Oxide team to stay aligned. We also collaboratively reviewed in-progress work instead of waiting for a polished final product. We maintained flexibility to focus on Oxide’s priorities and the next critical elements rather than following a linear, waterfall design approach. As Erik said, the transition between prototyping and development into production is always a challenge, but by maintaining transparency and using the development team’s experience to jumpstart the production team, we were able to meet Oxide’s schedule and assembly needs with their support at each phase.

Could you share some specific instances where early collaboration and continuous joint project reviews played a pivotal role in shaping the project’s trajectory?

Erik:

One thing that comes to mind immediately is the early feasibility study for blind mating our server sleds to the rack. This allows a customer to install or remove a server sled from the front of the rack without having to plug in or unplug any cables for either power or network connectivity. It took Benchmark about three months to do this study and the results gave us the confidence to commit to this differentiated feature for our product.

Amy:

Erik brought up a great example and one that really showed the benefit of each team leveraging their strengths to build a strong product. Another good example came from the EVT1 bring-up of the server component of the rack. Having Benchmark’s early build teamwork together with Oxide allowed us to fold in design changes and updates as we made them, helping us adapt during prototype builds and keep the mechanical, electrical, and software development efforts moving. By leveraging each team’s specific skills and expertise, we were able to complete EVT successfully and learn as much as possible during each build with minimal wasted time or materials.

Looking ahead, what lessons can other technology companies draw from this successful partnership in terms of fostering collaboration, innovation, and adaptability?

Erik:

We are proud to build our product at Benchmark Minnesota. Having most of our team located within one or two time zones of each other made it easier to communicate and therefore build trust. Given the strength of our partnership and Benchmark’s manufacturing excellence, we have confidence in our ability to grow our business together. 

Amy:

Some lessons for other technology companies to consider might include the importance of choosing a partner with relevant expertise and responsiveness, maintaining transparent communication, and employing strategies that foster adaptability and innovation, such as engaging suppliers early and allowing for rapid iteration and prototyping.

The Power of Collaboration and Innovation

The partnership between Oxide and Benchmark demonstrates the power of collaboration and innovation within the high-speed computing domain. By leveraging each company’s respective strengths, encouraging transparent communication, and developing the ability to adapt, the project continues to be a success. The two companies overcame several challenges together and created a rack-based server system in which both the hardware and software were co-designed. This unique partnership serves as an inspiration for others in the industry seeking to push the boundaries of technological possibilities through collaborative problem solving. Reach out to Benchmark to learn more.