Factory competitiveness can be enhanced through the use of cognitive support tools at all levels, especially for shop floor operators. To get the most value from implementing a tool it has to fit the needs of tasks, environment and users. The aim of the study presented here was to examine how a usability test of a high fidelity prototype can add value to the need analysis phase. Results show that when using the prototype as a mediating object it can provide new insights to further refine the need analysis, especially when using an environment of high ecological validity.