We were curious to know how companies implemented their design review process and what tools they used to achieve it. So we ran a survey over the last few weeks and talked to 400 engineers worldwide from all over the spectrum: big firms, startups, medium-size businesses, and so on. Surprisingly, we realized that in the era of advanced technology the most commonly used tools for design review were… highlighters, red pens, printed PDFs, and meeting rooms. One of the most elaborate devices used to complete design review must have been a projector or a laser pointer. Nothing really innovative there!
Having reflected on the survey, we came to the following conclusions.
1 – You already know the value of design review
Drawing a schematic is basically an exercise in carefully reading datasheets, app notes, and reference designs to learn how to best connect up each component for your application. This is in addition to all of the work involved in developing the top-level architecture for your board and selecting components to meet the product requirements. Each of these steps is an opportunity to make a mistake.
Every seasoned hardware engineer has had the experience of spending many frustrating hours debugging something on their prototype only to discover the root cause was a careless error in the schematic or layout that should have been caught earlier. This results in wasted time, a costly board spin, or worse, being completely blocked from testing a particular part of your board. Every hardware engineer knows that after spending many weeks working on the same schematic you’re simply unable to see errors any more. You need a fresh set of eyes to take a look. This is what design review delivers.
2 – There are all sorts of reasons that people fail at design review.
It’s well-known that design review isn’t done effectively right now. 75% of the engineers we spoke with told us they are well aware that they needed to do a better job at design review. Similarly, 71% of our beloved guinea-pigs described their process as ‘inefficient’, 23% found it truly ‘painful’, and only 6% were happy with their current tools and processes. The interesting question is why this is the case.
Our survey has revealed that: In order to really add value via design review you have to understand the product requirements, carefully read all of the datasheets/app notes/reference designs, understand what the designer is trying to accomplish, and then look for errors. If you don’t read the datasheets and get into the mind of the designer, you don’t add much value because then you’re only able to find the really obvious mistakes (like a power pin being unconnected), which are much less likely to be present. Design review requires you to spend a certain minimum amount of time which can easily be a few days for complex designs.
To that end a colleague is often asked to look at the design at the very end of the process, making the design appear large and daunting. As a result, the colleague, who is very busy him or herself, feels rushed and simply skims over the design and datasheets. They may not thoroughly focus on each section. Often the team culture doesn’t accept, nor encourage, an engineer to dedicate a few consecutive days to design review.
Another familiar scenario is a designer completing a schematic, sending it out to the team for review, scheduling a meeting with everyone a few days later, and everyone showing up to the meeting without having looked at the schematic! Everyone expected to talk about the design as a team. This approach is not as valuable. With layout it’s even worse. A completed layout, especially one with 8+ layers, can be so complicated that’s it’s too daunting and difficult to review all at once at the end. Finally, a lack of tools to streamline the process is the most prominent reason that companies skip design review or is the most likely reason for why they fail at design review (think back to those red pens and meeting rooms).
3- Design review shouldn’t be a separate task in the design cycle.
The best way to do design review, in our humble opinion, is to set up an incremental design review process. A short review should be done after each component (or small group of components) is placed on the schematic or layout and is mostly connected. This task is much more manageable for busy engineers. It may only take 1 hour or less to go over the datasheet and check for mistakes. It lets engineers ask questions early about design decisions. And best of all, when the design is finished and ready for final review, all team members are already familiar with every part of the design and will be much more effective at reviewing. In other words, we suggest distributing the review time throughout the design process instead of doing it all at the end. To dig into the analogy with software development, can you imagine writing an entire application and reviewing the whole code base at the end? This wouldn’t make much sense. And the claim that incremental design review works is not just a hypothesis: 97% of companies that implemented an incremental design review process in their standard design cycle found more defects on average than teams that did not. Even better, 72% had one less board revision.
Incremental design review needs to be instilled in the team culture. Managers need to foster an environment where mistakes are okay. Engineers shouldn’t feel uncomfortable putting their work up for incremental review. There needs to be a very supportive team culture where it’s understood that the primary goal is to create the best product possible and become better designers along the way. Mistakes should be celebrated (as long as they’re only made once) because you learn something really important with each mistake.
4 – Current design review tools may as well be from the Middle Age
Ok, this one may be a bit extreme but… red pens, and (non 3D) printed PDFs are probably the most inefficient tools to achieve a proper design review. Why? Because if you annotate and comment on printed PDFs then it’s likely that one poor dude will have to go back and fix all the defects. And he’s probably going to get several copies of the design with different issues pointed on each of them. Then he’s supposed to summarize all of it and adjust the design with as little delay as possible. With this approach, how do you understand design decisions? How do you keep track of what remains to be fixed? How does someone other than the overwhelmed designated-victim assess the project status and define an ETA?
But don’t panic, it is definitely not your fault. Nor your manager’s. There are currently no good tools for design review especially if you want to do incremental design review. Actually, that’s not true: there is now one tool to do this. Its called Upverter. But this is not the right time or place to talk about this 🙂