Blog

Why Autonomous Vehicles are Stuck in Neutral

Despite the surge in funding and news that autonomous vehicles (AVs) are attracting widespread adoption is still far from becoming a reality. To date, there has been a lot of success driving in relatively friendly conditions like highways in California. However, before AVs can be let loose in complex urban environments, they need to be much more competent, reliable, and safe!

The urban environment roadblocks

Unlike perfect highway driving conditions, AVs in an urban setting must drive within close proximity to other agents such as cars, bikes, and pedestrians, whose movements it does not control, and cannot completely predict in advance. An AV must be able to anticipate the potential future actions of other agents and preemptively maneuver to avoid collisions.

Cars-Swerving

Imagine a car driving at 25 miles an hour past a school that has just been let out for the day. The scene around it is filled with children of varying ages, many of them looking at their phones. If that car continued to drive straight at its current speed – which is the standard speed for urban roads – and only reacted when a child stepped onto the road, there is a high chance that it will be unable to stop in time. Similarly, any simple reactive reflex-like swerving may cause the car to hit another child on the sidewalk, or to swerve into the path of an oncoming vehicle. One safe alternative here is to assume the worst case: that all of the children could walk into the road at any minute; the right solution then is to halt the car in the middle of the street until all the children are gone, which is not practical.

AdobeStock 315443081-1

Instead, the car should sense the children, understand that there is a low but non-negligible probability that each of them may step into the road, and then proactively take action to avoid a collision by slowing down and creating some space between it and the sidewalk. This must happen before any of the children steps onto the road – afterward is already too late.

To do this requires the AV to model all the ways in which the agents around it may move, assign probability estimates to them, and then use those estimates to build a safe, proactive motion plan. But there are many agents, and many possible ways each of them can move, which makes the resulting problem very computationally challenging – even more so when safe operation requires the AV to plan several times a second, and react to unexpected events within 100ms. Existing software and GPU based motion planners can’t achieve this level of performance, and therefore can’t ensure an acceptable level of safety at normal driving speeds in urban environments.

The Realtime Factor

Here at Realtime, we have developed an AV motion planning processor that applies our deep technical expertise in designing specialized hardware for motion planning – we build the fastest motion planning solutions – to this critical problem, resulting in a power and cost-efficient motion planning unit capable of both proactive risk-preemption, and high-speed reactive response.

By leveraging precomputation, circuit-level parallelism, and high-speed specialized motion planning circuitry, we provide an innovative and unique solution. AVs with our technology can recognize and respond to unforeseen events, such as a pedestrian who suddenly steps off the sidewalk into the road. Another key advantage is that AVs are now able to react like a human without incurring any of our human failings, such as checking our phone while driving.

If fully autonomous vehicles are to become a reality in dynamic urban environments, then solving the motion planning conundrum is an essential piece of the puzzle. To find out more about how we are helping AV manufacturers then reach out here:”

Share on: Twitter | Facebook | LinkedIn

Realtime Robotics Raises $11.7 Million Series A Funding

Technology enables robots and autonomous vehicles to automatically plan and respond to changing environments

BOSTON (OCTOBER 16, 2019) – Realtime Robotics, the inventor of responsive motion planning for industrial robots and autonomous vehicles, announced today that it has raised $11.7 million in Series A funding. Led by SPARX Asset Management the round included participation from Mitsubishi Electric Corporation, Hyundai Motor Company, and OMRON Ventures, alongside existing investors Toyota AI Ventures, Scrum Ventures, and the Duke Angel Network. The new capital will be used to accelerate the development of more commercial product releases and expand the team to support key customers and partners across the globe.

The interest in the round reflects Realtime Robotics’ first-mover advantage in the market for solutions that eliminate the obstacles to widespread adoption of advanced automation in industrial, agriculture, food service, construction, healthcare, and consumer settings.

Despite the growing demand for automation, today’s robots are not safe or smart enough to navigate in dynamic, unstructured environments, without costly safeguards and oversight. Realtime Robotics’ solutions eliminate these challenges and enable robots to work at a productive pace. Its specialized computer processor and software enable machines, including industrial and collaborative robots and autonomous vehicles, to evaluate millions of alternative motion paths to avoid a collision and choose the optimal route before making a move, all in milliseconds. It released its first commercial solutions RapidPlan and RapidSense earlier this year and you can see the automatic planning technology in action:

Realtime Robotics was founded in 2016 by Duke University professors Dan Sorin, George Konidaris, and researchers Sean Murray and Will Floyd-Jones, based on groundbreaking DARPA-funded research in motion planning.

“The commitment garnered from strategic investors reflects both the need and the demand for smarter robots. Our technology transforms the way machines interact with both people and other machines.  Robots will now be able to take on a wide range of new tasks and manufacturers will finally benefit from the productivity and efficiency gains that increased automation has promised, but failed to deliver.”

Peter Howard | CEO | Realtime Robotics

“Realtime Robotics’ technology is ground-breaking and will transform the deployment of robotic automation across a range of industries. We’re proud to work closely with the team as they begin to help their customers reduce the friction, cost, and complexity of automation.”

Seiji Miyasaka | Head of US Investment Team | SPARX Asset Management Co., Ltd.

“We are excited to partner with Realtime Robotics to deliver the first fleet of robots that can work smartly and respond to changing environments on the fly. Our investment will accelerate the adoption of safe and productive robotic systems across the globe and will ensure that robots realize their potential in industrial settings and beyond.”

Satoshi Takeda | Senior Deputy General Manager of Nagoya Works | Mitsubishi Electric

“Dynamic motion planning in real time for unstructured environments has been a long-standing problem in robotics. OMRON Ventures is proud to support Realtime Robotics, whose innovations can vastly improve the capability and applicability of industrial robots leading to the creation of new business opportunities in this area.”

Tomoko Inoue | CEO | OMRON Ventures

About Realtime Robotics

Realtime Robotics has developed responsive motion planning solutions for industrial robots and autonomous vehicles. Its specialized processor generates safe robotic motion plans in milliseconds, enabling robots to function in unstructured, collaborative work-spaces, reacting to a dynamic world the instant changes are perceived. Its solutions expand the potential of automation. Learn more about Realtime Robotics here and connect on Twitter and LinkedIn.

Share on: Twitter | Facebook | LinkedIn

Realtime Robotics Unveils Realtime Controller

Simplifies and reduces the programming burden, accelerating the integration of robots

Realtime Controller

The Realtime Controller

BOSTON (April 30, 2020) – Realtime Robotics, the inventor of responsive motion planning for industrial robots and autonomous vehicles, today announced the launch of its Realtime Controller. The solution dramatically reduces and simplifies the programming required to safely integrate robotic workcells, speeding up the time to deployment.

As the pressure intensifies to improve margins it will accelerate the adoption of robotic automation. However, integrating robotic operating systems currently is complex, time-consuming, and cost-prohibitive, which has limited where and how the technology is deployed. With the Realtime Controller, multi-robot cells are much more flexible, as it automates many core processes and can dynamically adjust to variable production conditions.

Using Realtime’s Controller, manufacturers can now quickly and easily plan, simulate, and validate automation through the entire deployment. The Realtime Controller connects with the customer’s PLC and robot controller so that they can autonomously calculate, communicate, and execute collision-free motions. Both development time and cycle time are significantly reduced with automated interlocks and interference zone-free multi-robot workcells enabling robots to be deployed more quickly and expand into new areas that were previously cost-prohibitive.

The Realtime Controller is an industrial hardware computing platform with proprietary hardware and software that supports both off-line programming and run-time operation of single and multi-robot work cells. The software toolkit and API enable easy integration with PLCs, picking systems, robot controllers, simulation software, and other task planning solutions. Its web-based interface allows companies to configure and monitor robot workcells, including visualizing robot motions and managing fault conditions. In addition, Realtime’s motion planning enables immediate and autonomous fault recovery solutions for multi-robot workcells.

The Realtime Controller will be commercially available in North America, Europe, Japan and China beginning in May 2020.

“Our vision is to reduce the friction associated with deploying robots in industrial settings. The Realtime Controller is a catalyst for accelerating and simplifying robotic automation across the globe, enabling manufacturers to finally benefit from the power of industrial robots working together.”

 Peter Howard, CEO, Realtime Robotics

“Realtime’s RapidPlan technology and the new Realtime Controller provides the power to transform how we deploy and operate industrial robots. The solution dramatically reduces time for motion planning and eliminates interlocking, so our engineers can focus their efforts on designing fully capable work-cells and efficient multi-robot deployments.”

 Gurpreet Ghataore, Advanced Research Engineer, The Manufacturing Technology Centre

Other Resources

Share on: Twitter | Facebook | LinkedIn

How Realtime Reduces the Friction Associated with Deploying Industrial Robots

Instead of painstakingly programming each move and coordinating multiple robots to work together, Realtime takes command of the movement and coordination functions, autonomously moving single or multiple robots to their goals while avoiding both static and dynamic objects in the environment. The processor removes the need for manual iterative programming and a software toolkit allows users to intuitively work with the Realtime Controller to provide collision-free plans autonomously. This means you can expedite the entire deployment process — system engineering, motion planning, task deployment, and reprogramming — and combine them into a fully autonomous solution that saves you significant time and money while maximizing throughput.

Benefits for Manufacturers

With Realtime’s technology, manufacturers can transform the deployment of robotic automation.

  • Faster, easier robot programming with accelerated offline motion planning
  • Increased throughput with interlock-free multi-robot workcells
  • Flexible workcells with collision-free planning in real-time
  • Safely deploy industrial robots in shared workspaces typically reserved for collaborative solutions

If you are looking to transform how you program production cells and accelerate your automation implementation while maximizing flexibility, then Realtime’s innovative technology is the answer. Find out more about how Realtime is helping remove the friction associated with the deployment of multi-robot workcells in this white paper.

Share on: Twitter | Facebook | LinkedIn

Putting the Need for Interlocks in the Rearview Mirror

If one robot is good, then more robots should be better. In theory, putting several robots in a workspace, with all of them working simultaneously, should greatly boost productivity. In practice, however, the marginal benefits of adding robots are slim. Why? Programming! Engineers must orchestrate all of the robot motions to achieve as much performance as possible while guaranteeing that there will never be any collisions. This is extremely difficult — just imagine if you had to “program” a restaurant’s kitchen, orchestrating all of the motions of several chefs.

Because the problem of avoiding collisions is so difficult, robot programmers fall back on a simple, but poor performing solution. They identify every region in the workspace that is reachable by more than one robot, and they refer to these regions as interference zones. They then use locks to ensure that only one robot may be in an interference zone at any time, thus guaranteeing that collisions cannot occur. This solution is easy, but misses many opportunities for robots to be safely moving within different parts of an interference zone.

With technology from Realtime Robotics, multiple robots can productively share a workspace, with no complicated programming or interference zones. Realtime’s solution just requires the user to provide a list of tasks and task priorities, and this can be done with a simple PLC program without the need for interference zones or orchestration of any kind. From there, Realtime’s motion planning technology automatically finds collision-free motion plans for every robot, significantly reducing deployment time and cost, and significantly improving the productivity of the workspace.

Share on: Twitter | Facebook | LinkedIn

People and Robots, Working Together At Last

Today’s robot workspaces are designed to detect worker entrances—perhaps with a fence or a sensor—and which instructs all of the robots to idle. Why? Because today’s robots don’t perceive and react quickly enough at typical speeds to operate safely within close proximity to people. Having a fence or sensor that pauses the robot is the safest option.

Realtime Robotics has developed the Realtime Controller and RapidPlan which provide ultra-fast reaction time for robots. The core of the solution is real-time motion planning, which can quickly determine if a planned robot motion might cause a collision and immediately plan a different, non-colliding motion to get to the robot’s desired goal. Realtime’s technology empowers robots to react so quickly that they can easily continue to move—and be productive—in the presence of people.

There is no longer any reason for a workspace to sit idle while a person performs maintenance; the robots can just work around the person. In fact, truly collaborative robots or “cobots” are now possible, where people are actually expected to be in the workspace performing tasks alongside the robots.

To learn more about if Realtime’s solution is good for you, please contact us.

Share on: Twitter | Facebook | LinkedIn

Easy and Unobtrusive Failure Recovery

Unfortunately, robots don’t run smoothly all the time. Robot hardware or software could fail, an end-effector could break, or even some part of the workspace itself could fail (e.g., a conveyor belt providing materials to one of the robots). Failures cause a disproportionate amount of downtime.

To get back up and running, the operator must perform the following steps:

  1. First, the workstation must immediately cease activity to not cause further damage.
  2. The operator must manually reset every robot to its “home’ position.
  3. Remove any materials that had been incompletely processed at the time of stoppage.
  4. Perform the desired repair(s).
  5. Leave the workstation and restart the software that controls the workspace.
  6. Resume operations.

This is a long process that involves a significant amount of manual effort and impacts performance of the entire workspace. It is not possible to, say, replace the end effector of a robot and offload its work to another robot during this repair.

With technology from Realtime Robotics, failure recovery is fast, simple, and flexible. The operator enters the workspace and can decide whether or not to have the robots automatically return to their home positions. While the operator is performing the desired repair, the unaffected robots can continue to work, and there is no need to discard any in-progress work. Realtime’s technology also allows easy reassignment of tasks from an under-repair robot to other robots.

Get in touch to learn more about how we can help your robotic workstations.

Share on: Twitter | Facebook | LinkedIn

Realtime Robotics Partners with Siemens to Accelerate the Integration of Industrial Robotic Workcells

Simplified programming by automating motion planning and interlocks

BOSTON (June 25, 2020) – Realtime Robotics, the leader in autonomous motion planning for industrial robots, today announced a partnership with Siemens Digital Industries Software. The use of Realtime’s technology with Siemens’ Process Simulate software in the Tecnomatix® portfolio can help simplify robot programming and workcell coordination by automating motion programming. Manufacturers and integrators can now program, simulate, and validate automation, in efforts to simplify the digital commissioning process.

Currently, multi-robot workcells take prohibitive amounts of calendar and scarce engineering time in the simulation phase before going to production. Once deployed, they are inflexible, requiring significant re-programming for every modification. This has made multi-robot workcells cost-prohibitive for all but the most stable high-volume manufacturing.

Siemens’ industry-leading Tecnomatix® Process Simulate, coupled with the Realtime Controller and RapidPlan software, can significantly simplify the programming of automated robotic processes, helping reduce the time to deploy and adapt to changes, both during the simulation and on the shop floor. Pairing this technology helps eliminate the need to enter teach-points and to create and manage interference zones. Joint customer proof of concept projects shows programming cycle time reductions of over 70 percent, enabling quicker, more robust deployments.

“We are excited to announce our collaboration with Siemens. Coupling our technology solutions provides manufacturers with the flexibility and efficiency they have been clamoring for to expand the deployment of industrial automation. Together we will accelerate the pace of automation”

— Peter Howard | CEO | Realtime Robotics

“Our partnership with Realtime Robotics reflects a paradigm shift in the way robotic applications are programmed and deployed. Together we have created a unique value proposition that will open up a world of possibilities for robotic automation.”

— Zvi Feuer, Senior Vice President | Manufacturing Engineering Software | Siemens Digital Industries Software

About Realtime Robotics

Realtime Robotics has developed a specialized processor to generate safe motion plans in milliseconds for industrial robots and autonomous vehicles. Its solution enables robots to function together in unstructured and collaborative workspaces, as well as react to dynamic obstructions the instant changes are perceived. Its solutions expand the potential of automation. Learn more about Realtime Robotics here and connect on Twitter and LinkedIn.

Note: A list of relevant Siemens trademarks can be found here.

Share on: Twitter | Facebook | LinkedIn

Improving Bin Picking Performance

There is a great opportunity to have multiple robots work together in many application domains, including manufacturing and logistics. The potential performance advantage of being able to use multiple robots is similar to the benefits of being able to use multiple people to complete a task.

Using a standard bin picking application as a case study, we’re going to explain why current technology can’t take much advantage of this opportunity. But first let’s see where a single-robot gets bottlenecked, so that we better understand how multiple robots could provide benefit.

One Robot Per Task

Single Robot Bin Picking

This video shows a single robot picking parts out of an assembly bin and placing them in a package for shipment. Due to several performance bottlenecks, which are highlighted below, the robot performs this task more slowly than a person, and this poor performance limits the adoption of automation.

There are four performance limiting factors with one robot working on a task with conventional off-the-shelf methods:

  1. Physical limits: The time required to do each pick and place is based on the distance, speed and acceleration of the robot and processing data with conventional off-the-shelf methods. The most promising solution to increase performance is to add more robots, but this only helps if those robots can work collaboratively.
    (More on this later!)
  2. Gripper performance: There is some period of time during which the robot must stop moving in order for the gripper to open or close when picking and placing. As gripping technology continues to incrementally improve, gripping time will decrease.
  3. Perception performance: In this application a structured light camera is used to locate up to 8 parts inside the bin and provide pick poses for them. As camera technology improves, they will be able to provide targets and poses more quickly.
  4. Robot motion planning: When a pick pose is provided to a robot, there is some delay as the robot makes a motion plan to get to that pose. Motion planning has been a longstanding bottleneck in robotics, and it can take several seconds to perform using prior technology. Realtime Robotics has the only solution for eliminating this crucial bottleneck.
    (More on this later!)

Two Robots Per Task

Multi-Robot Bin Picking with Interlocks

In theory, adding a second robot should double the performance of the workcell, but practice and performance is far from theory. Instead of achieving ideal linear speedup as we add robots, we get the poor marginal gains of the “Current reality” blue curve.

Why is the marginal benefit of the second robot so limited? Because the robot motions must be coordinated to not collide with each other when picking. This process is extremely challenging for robot programmers, which is why the industry standard solution is conservative and leaves a lot of performance on the table.

rtr-chart-performance

The current industry approach is to introduce the concept of an interference zone, or an area where only one robot is allowed to be at any time. In a workcell, any region that is reachable by more than one robot is its own interference zone. In our example, an interference zone is designated above the blue bin, meaning only one robot can be above or in the bin at one time. This ensures the robots will not collide with each other, because while the first robot is picking the second robot is waiting outside the interference zone.

Specifically, even if more than one robot could have safely been in separate parts of the interference zone, that would be prohibited. In our application, one robot idles while the other robot makes a pick.

Two Robots with Realtime’s Technology per Task

Multi-Robots Bin Picking with Realtime Technology

The key to optimal performance is having both robots work simultaneously. The Realtime Controller actively monitors the robots and the workspace to generate collision-free plans in real-time. With both robots being able to have safe, close movements the need for conventional interference zones is eliminated. The result is dramatically increased throughput, because robots are not needing to idle when the other is within the interference zone.

The Upshot: How Much More Performance?

For specific performance impact on how much performance was gained, we ran four trials with each scenario. The table below shows the average performance of all the trials. The Realtime Controller achieves a 74% higher performance than the single-robot baseline, whereas using interference zones achieves only 14% higher performance.

Bin Picking Performance Comparison Chart

Share on: Twitter | Facebook | LinkedIn

Transforming Industrial Automation with the Rapid Motion Planning, Pt 1

Why Rapid Motion Planning?

Realtime Robotics has developed the Realtime Controller that allows you to quickly deploy a multi-robot workcell that maximizes the collective performance of the robots. In the video on the left, you see four robots—three from Universal Robots and one from Fanuc—moving together in a demonstration workcell. All of the robots are moving quickly to perform simulated tasks, rarely slowing or stopping to avoid collisions with each other. The video makes this look deceptively easy, but this video shows a capability that is far beyond the state-of-the-art in industrial automation.

If you’ve ever had the pleasure of designing and deploying a multi-robot workcell, you know there have been two major pain points:

1. Workcell design has been slow and costly

It takes a lot of time and engineering effort to design a multi-robot workcell. There are many possible ways to position the robots. There are also many possible ways to assign tasks to robots and to order the tasks that are assigned to each robot. The combined possibilities for robot placement and task planning are vast, and manually searching for good solutions requires extensive engineering time and effort. Realtime has developed an automated tool that streamlines the process of finding optimized solutions, but that is the subject of another blog post; for now, assume the robots have been positioned and each robot has an ordered list of tasks to perform.

2. Robot motion planning has been slow and inefficient

All of the robot motions must be programmed to avoid collisions. Robot motion planning has been a longstanding bottleneck in robotics, and it is a painstaking process to determine every robot’s motion. To simplify the process, at the expense of performance, robot programmers frequently designate areas that can be reached by more than one robot as “interference zones”; by only permitting one robot in an interference zone at a time, collisions are avoided. This solution leaves significant performance on the table, because it is often the case that multiple robots could be in different parts of an interference zone at once, and the use of exclusive interference zones eliminates this productive activity.

The Realtime Controller solves the motion planning problem. With special-purpose hardware and a software toolkit, the Realtime Controller can perform hundreds to thousands of motion plans per second, which is orders of magnitude faster than the state-of-the-art. At that speed, it can quickly find optimized motion plans for every robot, guaranteeing there will be no collisions, without requiring interference zones. And, should the work cell change a bit—say, by moving a robot or changing the set of tasks to be performed—the Realtime Controller can quickly generate new motion plans without having to wait for weeks while engineers re-choreograph all of the motions. Robot programming is now just a simple matter of telling the robot where to end up; the Realtime Controller handles all of the complexity of figuring out how to get it there safely. Furthermore, the Realtime Controller can do this for all major robot OEMs.

What does this all mean for industrial automation? You can now deploy higher-performance work cells with your choice of robots in far less time!

Get in touch to learn more about how our rapid motion planning solution can help your operations.

Share on: Twitter | Facebook | LinkedIn