We test all vacuum cleaners on both hardwood floors and carpet.
How we test robot vacuum cleaners
Our method for evaluating robot vacuums is simple, but grueling. We perform two types of tests. The first test is to find out how well a robot covers the floor while it cleans. We have built an industry standard test room as specified by the International Electrotechnical Commission for this purpose only. The IEC is an international standards body responsible for managing robotic vacuum test procedures, among others, for vacuum manufacturers.
Say hello to our robot vacuum test room. Inside are objects that may look a little strange. They are designed to simulate furniture and obstacles that a robot would encounter in the real world.
Within this chamber are objects designed to simulate typical obstacles a robot vacuum encounters as it navigates through a cleaning job. These obstacles include wall edges, table and chair legs, couches and other furniture, and so on, plus bare tile and hardwood floors, as well as carpeting. We mount LED lights on the top of each vacuum. The dimensions of the lights correspond to the measured nozzle width of each specific robot vacuum we test.
As the robots move around the room while cleaning, an overhead camera captures a long-exposure image of the entire room in low light. That image will then have a light trail created by the LEDs that shows the exact areas the robot passed through (and the position of the nozzle) during its runtime. We can also see areas of the floor where the vacuum may have missed or gotten stuck. You can see the navigation results of all the robot vacuums in our test group in the gallery below.
The second type of test shows exactly how much physical dirt a vacuum can suck up from the floor. To simulate dirt of small particle size, we use a mixture of play sand and landscape sand. For soil with larger particles, we use grains of uncooked black rice. Robots then run in a straight line over three types of floors (low pile carpet, medium pile carpet and bare hardwood floors).
More from our robot vacuum cleaner test setup.
We also check the specific nozzle width of each vacuum. We built an adjustable tool to soil our test floors. This allows us to lay down a strip of soil of a precise area that matches the nozzle size for each robot. The mass of soil is not chosen randomly either. We measure a proportional amount that is related to the floor material, the type of dirt, and the nozzle width of each vacuum.
We perform a minimum of three cleaning rounds on each floor type. We also perform cleaning tests with sand and rice separately. That amounts to a minimum of 18 tests per robot vacuum. We weigh the robot’s dust bin both before and after each round. From there, we can calculate the percentage of dirt vacuumed for each cleaning round and the average amount of dirt a machine is able to remove. In addition, we perform anecdotal (visual) pet hair tests for each robot, on all three floor types.
Our test with rice-based, medium-sized particles didn’t show enough differentiation between each cleaner, saying they can all handle larger particles without issue. For pet hair removal, we’ve gone anecdotally.
How we test cordless vacuum cleaners
We perform straight line testing on all three floor types.
Testing cordless vacuums isn’t as complicated as testing a robot vacuum, but it still takes a lot of time and careful effort to find the best cordless vacuum. We run each vacuum in a straight line across three different surfaces (hardwood, low-pile carpet, medium-pile carpet). On all three test beds, the test area is the same length (30.25 inches).
We tested how well a vacuum cleaner can suck up sand and rice.
As with robot tests, the width of the test bed is proportional to the width of the vacuum cleaner’s nozzle. We measure this width ourselves. We also use the nozzle width, plus the type of floor, to calculate the soil density for each test, according to IEC guidelines. We also use the same types of floor here; sand, rice and animal hair. We perform three runs (minimum) on each floor type. We also test the suction power with sand and rice separately. That amounts to a minimum of 18 tests per vacuum cleaner. We weigh the vacuum cleaner’s dust container both before and after each run.
From there, we can calculate the percentage of dirt and debris that is sucked up for each run, and the average amount of dirt that a vacuum is able to remove. Additionally, we perform anecdotal (visual) pet hair tests for each vacuum, on all three floor types to help us select the best cordless vacuum.
