A more important reason why judges at the Senior Design Conference will be looking for qualitative results gathered from prototype tests is that engineers are always responsible for ensuring the safety and effectiveness of their designs. While judges aren’t necessarily looking for industry-level analyses or performance specifications from your team, they are going to look at your team’s efforts to back up your design claims with relevant data that your team gathers from controlled experiments. The time between the prototyping phase and the Senior Design Conference, however, often leaves teams hard-pressed to find enough time to conduct validating tests on their prototype.
Engineering design, at its core, is a people’s service and looks to provide technological solutions to safely and effectively assist people with real-world problems. The design project you and your team choose is not excluded from this, which demands that your team is able to produce qualitative and quantitative justifications on why your project is able to meet expectations of safety and performance. In my own experience working to complete a testable prototype for review at the Senior Design Conference, I found that both the Rights and Fairness and Justice Approaches were most helpful. Design validation through use of the scientific method in testing is a crucial part of honoring engineering Virtue Ethics, and demands engineers to employ a Rights Approach in respecting customers’ rights to a safe and effective product.
What does prototype testing have to do with rights or virtues?
At the beginning of the Code of Ethics of the American Society of Mechanical Engineers, the first fundamental canon states: “Engineers shall hold paramount the safety, health, and welfare of the public in the performance of their professional duties.”(American Society of Mechanical Engineers, 1976) [3]. ASME, ASCE, IEEE, and many other national and international ASME, ASCE, IEEE, and many other national and international orders of engineers have similar codes of ethics that, altogether, state that the core virtue of all engineers should be the respect for life and the safety of the public. Upholding this virtue, then, comes with ensuring the safety and effectiveness of engineering designs.
But how can engineers guarantee a reasonable level of safety in the use of their designs? Good and safe products are always tested as thoroughly as possible in the design process well before deployment. Even in very expensive products that may be too costly or infeasible to test the entirety of the structure, efforts are made to validate the design on the granular level to determine the safety of individual design components through stress calculations, finite element analysis (FEA), and other applications of failure theories.
Because of the need to assess the safety of an engineering design with rigorous scientific methods and statistical confidence, all codes of ethics contain a statement to the effect that “engineers shall be truthful in explaining their work and merit, and shall avoid any act tending to promote their own interest at the expense of the integrity and honor of their profession or another individual.” [3]. Prototype Validation not only asks engineers to hold themselves to virtues of integrity and reliability-- it also meets the rights of consumers to have a “justified claim” on engineers to satisfy their right to safe and effective products. This demand for consumer respect imposes a “duty to help sustain the welfare of those who are in need of help” on all engineers, which is held explicitly in engineering codes of ethics (Velasquez et. al, 1990) [4].
As students looking to graduate and enter the engineering profession, you are not exempt from these expectations during the senior design project. The design choices you and your team make during the senior design project are much more than a matter of completing an undergraduate requirement. They will directly reflect the professional virtues that you and your teammates are developing well before working as a professional engineer. This is especially true in prototype and conceptual design testing and validation, where your team is the only group of people responsible for careful and honest data collection, analysis, and reporting.
Experimental design, data collection, and analysis place the responsibility of design validation into your hands. You and your team have an inherent responsibility to ensure that your methods of testing and presentation of data regarding your prototype agree with the virtues of honesty, reliability, and fairness that are expected of engineers.
Integrity in Experimental Design, Data Analysis & Reporting of Results
Ensuring the highest quality and level of safety possible of engineering designs calls upon the integrity of the engineer to both honestly report their data and conduct well designed, repeated tests. What constitutes the “right tests” varies from project to project, but the goal of testing should be to ensure that product design specifications are met as safely and efficiently as possible.
In order for testing to have any real value, however, experiments must be designed to be repeatable and with scientifically appropriate metrics. Tests should be done with comparison to a datum or control of reasonable quality to ensure that the data of your design is fairly obtained. Finally, instrumentation and testing environments that produce as close to accurate results as possible should be used to report data to a degree that can validate the design with a reasonable quality. Altogether, applying good scientific methods and reporting collected data with explicit honesty are fundamental practices that mark integrity in an engineer.
Some helpful ethical considerations that can ensure that your efforts to validate the design align with engineering expectations of integrity include:
- Repeatability:
- Some projects, like satellite missions or rocket launches, may not be able to be tested multiple times. Can the tests you are looking to conduct be repeated? If so, how many times? Is this enough times to report data with statistical significance and confidence?
- Statistical significance is what backs engineering claims to safety and effectiveness. If tests cannot be repeated for a sufficient amount of data, do these tests still produce viable and significant results to validate your design?
- Datum & Controls:
- Are the tests that you are conducting being compared to a specific datum, or control? If so, will the datum and control be held constant throughout experimentation?
- Are the datum/control of a reasonable level of comparison to your device? For example, if you are testing a carbon fiber bike frame, are you comparing your design to data or control testing specimens of reasonable quality?
- Testing Conditions:
- Your product may be designed to operate in a specific environment that is not similar to what is present at SCU. If so, can this environment be reasonably replicated to test your prototype? If not, do tests outside of this environment still produce viable and significant results to validate your design?
- Testing Equipment:
- Many projects will require a high degree of precision in measurements and fabrication. Does the equipment that you are using to test your prototype have the ability to conduct measurements at a reasonable level of precision? If not, what is an acceptable margin of error that can still be used to produce viable and significant results to validate your design?
- Organic subjects/human subjects protocol:
- Are there living things and/or human beings that are necessary to conduct the tests needed to validate your design? Does your team have an experimental protocol that can confirm with reasonable certainty the safety of the subjects involved? If not, how will your team adapt the experiment to be safer for organic subjects?
Some helpful ethical considerations that can ensure you and your team accurately and honestly convey the data from your testing include:
- Statistical significance:
- What do your numbers say? Does your data establish a clear enough trend that allows your team to report positively on the safety and effectiveness of your prototype?
- For one reason or another, your tests may not produce data that agrees with your expectations of the design. If your numbers do not confirm that your design is safe, or that does not meet product design specifications, how will your team report the data?
- If there is not enough time to produce a statistically significant amount of data, is it to your team’s standards to conclude on your results with confidence? If not, what results can your team report, and how will your team do so?
- Assumptions:
- For a good amount of engineering calculations, there are reasonable assumptions that can be made to simplify the math, or more easily model a component. What are some assumptions or things deemed negligible in your team’s data analysis? Can these assumptions be reasonably made, and still produce viable and significant results to validate your design?
- Are the assumptions that your team is making accounted for in your design calculations? If so, how is your team stating these assumptions in your analyses and discussion?
- Translation to the customer:
- Your customers will most likely not have the same technical knowledge and understanding that your team will-- and will most likely react most to whether or not your design does what it is expected to do, and if it is safe to use. How can your team most accurately and most clearly express your findings to your customer? If they are not engineers, technically minded, or not educated on the material related to your project, how can your data be reported to address your customers’ concerns for safety and effectiveness of your design?
Verification of Results, Deployment, & Customers’ Right to Safe Products
Reporting the results of your experiments accurately and honestly is part of upholding one’s call to integrity as a practicing engineer. What engineers do with these results to continue to iterate and refine their designs for their customers, however, is also about the rights of the customer. Because engineering is a people’s service, the customer has a “justified claim” on engineers to only deploy products that can assuredly “sustain the welfare of those who are in need of help”(Velasquez et al, 1990) [4]. In short, your customers have the right to have as close to a finished product as your team can produce. This means that your design has been prototyped and iterated well enough to confirm with a high level of confidence that your design can operate safely and effectively to meet customer needs.
One thing that your team may find in the Senior Design project is that the completion of your prototype does NOT mean that your team has produced a minimum viable product (MVP) that is ready for deployment. The senior design project is so short compared to industry engineering projects that your team may not be able to have a working prototype ready for deployment. Even then, it is difficult to have a prototype that achieves its specifications without presenting significant risk to the customer and/or public.
Your team should collect feedback on your design, but only with a reasonable level of certainty that your design will not present any significant risk to them. It is your job, as the engineer, to first gather an appropriate amount of data to verify the safety and effectiveness of the design. Then, using your best judgment, decide whether or not your design is ready to be tested and deployed to your target customer or community for implementation.
Some helpful considerations that can assist your team in determining whether or not your prototype is ready for deployment to your customer or community include:
- Experimental Protocol & Safety Hazards Analysis:
- What is the safest way to operate/test your device? How will your team best communicate an experimental protocol or operation manual to instruct the customer how to safely use the device?
- What risks are present in the construction/assembly/operation/testing of your design? How is your team minimizing the risks and how will the customer become aware of these risks?
- Does your prototype present any immediate safety hazards, such as exposed gears, pinch points, high voltage drops/currents, sharp edges, etc? What engineering standards can be used to best address those safety hazards prior to deployment (e.g, OSHA for gears, belts, chains, ASME boiler codes, ESFI codes)?
- Results of Data & Inconclusivity:
- Gather the results from your experiments and compare the data to your product design specifications. Does your data confirm the safety and effectiveness in the construction/assembly/operation/testing of your design? If not, how will this affect the schedule of iteration and deployment of your prototype? How will this information be conveyed to the customer to best respect their right to updates on the progress of the design?
- Your tests may not provide a definite answer on the safety or effectiveness of the design. How will your team move forward as to best improve the design? How will this information be conveyed to the customer to best respect their right to updates on the progress of the design?
Personal Link:
Although it became clear at the beginning of our spring quarter of senior design that my team and I would not be able to return to Nicaragua in the summer to deploy our manual clay press prototype, we continued to work towards finalizing a standing prototype of our design. This was important to our team for two main reasons. Firstly, our team wanted to be able to collect data for completion of our senior thesis and for future design iterations. Our tests determined that while the prototype was able to compress clay and produce stronger bricks, there were several mechanical interactions in the assembly that were too tightly toleranced. Our Compression Chamber, for example, was able to slide into and out of our Base Fixture for compression operations, but not without having to manually reorient the chamber along the steel retrieval rails. Our Compression Plate, which descended into the Compression Chamber to achieve compression, was also prone to tilting if it entered at an odd angle, which would sometimes cause unequal compression of the clay inside the Compression Chamber. Observations of these interactions not only helped complete our thesis, but allowed us to mark several changes that needed to be made to our design before implementation. Moreover, our tests allowed us to have results to communicate with the owner of the brick making coalition in our target community in Nicaragua. We understood that as our customer, he had the right to expect our team to have made a significant effort to produce a safe and robust design-- and that he also had the right to understand why the prototype would not be not deployed in the summer of 2018.
Figure 3: The standing prototype of the Frugal Clay Press for Nicaragua that was tested and demonstrated at the Senior Design Conference on May 10, 2018.
Lastly, our team collected data on our prototype to provide future design teams with adequate data to iterate effectively upon our design. We made an extensive effort to document the performance of the prototype and list the key interactions in the assembly that required a significant redesign. After the Senior Design Conference, my team contacted SCU faculty in the Department of Mechanical Engineering as well as rising seniors in mechanical engineering about our project. We established our design project as a legacy project which would be carried on and completed by another senior design team. This was important to each member of my design team on both a personal and professional level. We did not want our design project to end unfinished, and in doing so, disregard our customers’ right to the effective design solution they were promised. By establishing the Frugal Clay Press for Nicaragua legacy project, our team laid the foundation for a future team to complete our design and ultimately ensure the delivery of a more effective product to our customer in Nicaragua.
References:
[3] “Code of Ethics.” ASME Colorado Section, The American Society of Mechanical Engineers, 7 Mar. 1976, https://asme.org/
[4] Velasquez, Manuel, et al. “Rights.” Markkula Center for Applied Ethics, Markkula Center for Applied Ethics, 8 Aug. 2014.