MSGID: 1:317/3 64927d13
PID: hpt/lnx 1.9.0-cur 2019-01-08
TID: hpt/lnx 1.9.0-cur 2019-01-08
 Researchers expand ability of robots to learn from videos 
 Robots able to accomplish tasks after watching people perform them in any
environment 

  Date:
      June 20, 2023
  Source:
      Carnegie Mellon University
  Summary:
      New work has enabled robots to learn household chores by
      watching videos of people performing everyday tasks in their
      homes. Vision-Robotics Bridge, or VRB for short, uses the
      concept of affordances to teach the robot how to interact with
      an object. Affordances have their roots in psychology and refer
      to what an environment offers an individual. The concept has
      been extended to design and human-computer interaction to refer
      to potential actions perceived by an individual. With VRB, two
      robots successfully learned 12 tasks including opening a drawer,
      oven door and lid; taking a pot off the stove; and picking up a
      telephone, vegetable or can of soup.


      Facebook Twitter Pinterest LinkedIN Email

==========================================================================
FULL STORY
==========================================================================
New work from Carnegie Mellon University has enabled robots to learn
household chores by watching videos of people performing everyday tasks
in their homes.

The research could help improve the utility of robots in the home,
allowing them to assist people with tasks like cooking and cleaning. Two
robots successfully learned 12 tasks including opening a drawer, oven
door and lid; taking a pot off the stove; and picking up a telephone,
vegetable or can of soup.

"The robot can learn where and how humans interact with different objects
through watching videos," said Deepak Pathak, an assistant professor
in the Robotics Institute at CMU's School of Computer Science. "From
this knowledge, we can train a model that enables two robots to
complete similar tasks in varied environments."  Current methods of
training robots require either the manual demonstration of tasks by
humans or extensive training in a simulated environment. Both are time
consuming and prone to failure. Past research by Pathak and his students
demonstrated a novel method in which robots learn from observing humans
complete tasks. However, WHIRL, short for In-the-Wild Human Imitating
Robot Learning, required the human to complete the task in the same
environment as the robot.

Pathak's latest work, Vision-Robotics Bridge, or VRB for short, builds
on and improves WHIRL. The new model eliminates the necessity of human
demonstrations as well as the need for the robot to operate within an
identical environment.

Like WHIRL, the robot still requires practice to master a task. The
team's research showed it can learn a new task in as little as 25 minutes.

"We were able to take robots around campus and do all sorts of tasks,"
said Shikhar Bahl, a Ph.D. student in robotics. "Robots can use this
model to curiously explore the world around them. Instead of just
flailing its arms, a robot can be more direct with how it interacts."
To teach the robot how to interact with an object, the team applied the
concept of affordances. Affordances have their roots in psychology and
refer to what an environment offers an individual. The concept has been
extended to design and human-computer interaction to refer to potential
actions perceived by an individual.

For VRB, affordances define where and how a robot might interact with
an object based on human behavior. For example, as a robot watches a
human open a drawer, it identifies the contact points -- the handle --
and the direction of the drawer's movement -- straight out from the
starting location. After watching several videos of humans opening
drawers, the robot can determine how to open any drawer.

The team used videos from large datasets such as Ego4D and Epic
Kitchens. Ego4D has nearly 4,000 hours of egocentric videos of daily
activities from across the world. Researchers at CMU helped collect some
of these videos. Epic Kitchens features similar videos capturing cooking,
cleaning and other kitchen tasks.

Both datasets are intended to help train computer vision models.

"We are using these datasets in a new and different way," Bahl said. "This
work could enable robots to learn from the vast amount of internet
and YouTube videos available."  More information is available on the
project's website and in a paper presented in June at the Conference on
Vision and Pattern Recognition.

    * RELATED_TOPICS
          o Health_&_Medicine
                # Medical_Education_and_Training # Workplace_Health #
                Medical_Devices # Human_Biology # Medical_Topics #
                Infant's_Health # Staying_Healthy # Elder_Care
    * RELATED_TERMS
          o Robotic_surgery o Nanorobotics o Human_cloning
          o Personalized_medicine o Therapy_dog o Tattoo o
          Transmission_(medicine) o Placebo_effect

==========================================================================
Story Source: Materials provided by Carnegie_Mellon_University. Original
written by Aaron Aupperlee. Note: Content may be edited for style
and length.


==========================================================================


Link to news story:
https://www.sciencedaily.com/releases/2023/06/230620113807.htm

--- up 1 year, 16 weeks, 1 day, 10 hours, 50 minutes
 * Origin: -=> Castle Rock BBS <=- Now Husky HPT Powered! (1:317/3)
SEEN-BY: 15/0 106/201 114/705 123/120 153/7715 218/700 226/30 227/114
SEEN-BY: 229/110 112 113 307 317 400 426 428 470 664 700 291/111 292/854
SEEN-BY: 298/25 305/3 317/3 320/219 396/45 5075/35
PATH: 317/3 229/426