L'integrazione Di Robotica E Visione Per Il Flexible Manufacturing
This document was uploaded by user and they confirmed that they have the permission to share it. If you are author or own the copyright of this book, please report to us by using this DMCA report form. Report DMCA
Overview
Download & View L'integrazione Di Robotica E Visione Per Il Flexible Manufacturing as PDF for free.
More details
- Words: 3,159
- Pages: 35
QuickTime™ and a decompressor are needed to see this picture.
Tight Integration of Machine Vision & Robotics with LabVIEW in flexible manufacturing, automated assembly, packaging and testing Ignazio E M Piacentini [email protected] www.imaginglab.it NIDay 2009, Milan, 25th February 2009
QuickTime™ and a decompressor are needed to see this picture.
Agenda
The machine vision and robotics market: few numbers What is Flexible Manufacturing? Vision and robotics in flexible manufacturing: an example Basic concepts and requirements Few application examples Standards in Machine Vision / lack of standards in robotics Direct communication from LabVIEW to the robots: introducing a robotics library / a robotics toolkit in LabVIEW Questions and answers
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
2
QuickTime™ and a decompressor are needed to see this picture.
Machine Vision and Robotics is a growing mkt
There are plenty of new areas for robots, especially low payload, high accuracy ones Today only 5-10 % of robotics apps make use of MV, but these number is destined to grow: 35% by 2012 ??
Estimated yearly supply of industrial robots in Europe by countries 2005 - 2006 Germany Italy France Spain BENELUX
2006 2005
Central/Eastern Europe
100.000 robots deployed annually around the world 4000 – 6000 robots are deployed in Italy
NIDay, Milan, 25-02-09
United Kingdom Sweden Austria Switzerland Denmark Turkey all other countries
0
2.000
4.000
6.000 units
Tight integration of vision & robotics
8.000
10.000
12.000 Source:World Robotics 2007
3
QuickTime™ and a decompressor are needed to see this picture.
Italy: 'meccanica e macchine' : the weight of Italy in Europe
The weight of Italy in Europe: GDP (GEP?), manufacturing industry, ‘machines’ production Italy ranks at the 4th place, but machine manufacturing is 2nd after Germany
Italians are pretty good at building and exporting ‘macchine speciali’
Weight of Italy in Europe
Germany
UK
France
Italy
Spain
Other EU members
% of Gross European Product
20.0%
16.4%
15.4%
12.8%
8.5%
26.9%
Manufacturing industry
26.0%
11.3%
15.1%
14.2%
7.5%
25.9%
Machine manufacturing
33.4%
9.4%
11.1%
19.5%
5.0%
21.6%
Source: Federmacchine / Eurostat NIDay, Milan, 25-02-09
Tight integration of vision & robotics
4
QuickTime™ and a decompressor are needed to see this picture.
'Macchine speciali', vision, robotics, automation and LabVIEW Just one LabVIEW - NI platform for:
Machine Vision with the Vision Developer Module Robot programming with the ImagingLab robotics library Remote I/O and machine automation with C-RIO / PACs / PLCs HMI and remote machine monitoring
“ LabVIEW Everywhere ! ” NIDay, Milan, 25-02-09
Tight integration of vision & robotics
5
QuickTime™ and a decompressor are needed to see this picture.
Machine vision and robotics: tight integration
Robot application without vision usually require simple programming Very simple applications can be programmed directly from the ‘teaching pendant’ When vision is present, programming gets more complex (dynamic missions / dynamic programming) In machine vision + robotics application vision should not be considered an ‘add-on’, but designed / programmed as a whole unit and possibly in a single programming environment
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
6
QuickTime™ and a decompressor are needed to see this picture.
Why and what is 'flexible manufacturing'? 1: market changes
Market and economics changes have a deep effect on engineering Production methods and requirements have changed a lot in recent years:
Flexibility vs production speed
Smaller production lots Faster time to market Lowering production costs to respond to agressive new competitors (China, India mkts) Cycle time: what does it really means? Design for speed Design for flexibility Design for ease of use Design for low cost of ownership
Pictures courtesy of Automatismi Brazzale and Cabur NIDay, Milan, 25-02-09
Tight integration of vision & robotics
7
QuickTime™ and a decompressor are needed to see this picture.
Why and what is 'flexible manufacturing'? 2: robotics
Assembly machines need to be able to produce different objects / products:
Flexible automated assembly:
At least in the same class of products No re-tooling required Fast production lot change with limited or no operator intervention Used by non skilled personnel Reconfigurable / reusable for new products In-process quality control (as opposed to end-ofline quality control) Loading / feeding components Putting the components together
Flexible and reconfigurable ‘mechanics’ for handling and assembly Highly modular The answer is: more robotics NIDay, Milan, 25-02-09
Tight integration of vision & robotics
8
QuickTime™ and a decompressor are needed to see this picture.
Why and what is 'flexible manufacturing'? 3: machine vision
The robots need to perform more complex and ‘dynamic’ (i.e.) work cycles Robots need to see! Vision is used to: Locate the parts and send info to the robot Perform quality inspection of single components Verify assembly correctness
In-process vs end of line quality control:
Discards only defective component Prevents machine malfunctioning / jamming
The answer is: more vision
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
9
QuickTime™ and a decompressor are needed to see this picture.
Why and what is 'flexible manufacturing'? 4: better integrated software
Machines need to be reconfigurable by the end user Vision and robotics sw needs to have simple user interface The integration of robotics and vision must take care of calibration procedures, machine commissioning and start-up tools, etc The answer is: high level software libraries and tools specific to vision and robotics integration
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
10
QuickTime™ and a decompressor are needed to see this picture.
Flexible manufacturing / flexible feeding: the feeding station
Conventional vibrating bowl-feeder are adjusted for a specific component: lack of flexibility!! Flex feeders: computer-controlled vibration amplitude, direction and frequency Machine vision identifies the components Robots pick and place the part LabVIEW offers one single software platform Advanced tools for auto-calibration Advanced tools for system configuration Quality control of the selected parts NIDay, Milan, 25-02-09
Tight integration of vision & robotics
11
QuickTime™ and a decompressor are needed to see this picture.
Flexible assembly on the factory floor
Co-design and collaboration is a key to success (ImagingLab, Automatismi Brazzale, Cabur) From the early prototype to the deployed machine A short video-clip …
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
12
QuickTime™ and a decompressor are needed to see this picture.
Customers and applications Our customers are small / medium companies, often with strong mechanical background …or large companies / institutions From neutrino detectors, to semiconductors, to packaging, to cosmetics, … Automated assembly, testing, packaging, … Some of the most interesting applications are covered by NDAs and cannot be shown in this presentation We will see a limited number of video-clips All projects are based on LabVIEW + IMAQ + ImagingLab robotics library The robotics library for LabVIEW has been deployed on approx 40 different robotics / vision installations
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
13
QuickTime™ and a decompressor are needed to see this picture.
The essence of vision-robotics integration: the equivalent of LabVIEW 3-icon demo
Acquire the image Extract the information Download info to robot & execute
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
14
QuickTime™ and a decompressor are needed to see this picture.
Machine vision, GPM, and ….
We will not enter into the details of machine vision but .. Geometric and conventional pattern matching need to be complemented by other checks Parts may have different orientations / views It is necessary to discriminate recognizable / pickable / conformal (QC) parts
Some inherent complexity can be removed with a ‘smart’ user interface The end user must be able to set pass / fail criteria, vision and robotics origin and zeros Auto-calibration tools are a must NIDay, Milan, 25-02-09
Tight integration of vision & robotics
15
QuickTime™ and a decompressor are needed to see this picture.
The vision platform: from panel PC to Smart Cameras
The hardware selection is dictated by the application needs:
Conventional PC Industrial Panel PC PXI-based industrial PC (Win-based or RT OS) Compact Vision System (Real Time OS) Smart Camera (Real Time OS)
The software platform remains the same!! See Smart Cam videoclip
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
16
QuickTime™ and a decompressor are needed to see this picture.
A complex application: testing DIMM memory modules @ EEMS (ex Texas Instruments) (1’ 35’’ movie)
NIDay, Milan, 25-02-09
Fully automated functional testing of DIMM memory modules 64 interchangeable motherboards/ 256 memory slots 2 robots with common working space (collision avoidance) Full auto-guide / auto-learn / onboard camera Total testing cycle time compatible with production line Repetitive project
Tight integration of vision & robotics
17
QuickTime™ and a decompressor are needed to see this picture.
‘Smart’ pick and place for the cosmetic packaging
Fast picking from conveyor and accurate placement (< 0.1 mm) Many different sizes and shapes require flexible imaging configuration tool Most repetitive project using inverted Denso SCARA (22 robots, more to come ….) Triggered the development of the direct LabVIEW > b-Cap library Co-design with partner company NIDay, Milan, 25-02-09
Tight integration of vision & robotics
18
QuickTime™ and a decompressor are needed to see this picture.
‘Smart’ pick and place for the cosmetic packaging /2
Adding the brush to face-powder kit An early testing video-clip
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
19
QuickTime™ and a decompressor are needed to see this picture.
EST Projet (Educare per la Scienza e la Tecnologia): teaching vision & robotics at the ‘Leonardo da Vinci’ Science Museum in Milan Un progetto, anzi due, per il Museo della Scienza di Milano
NIDay, Milan, 25-02-09
Two ‘machines’ sponsored by NI, Mitsubishi, ImagingLab Recognising shape and colour, assembling and disassembling Interactive Taking care of an audience of variable age / designing for zero or little maintenance
Tight integration of vision & robotics
20
QuickTime™ and a decompressor are needed to see this picture.
Conveyor tracking in pharmaceutical packaging
Dual line with 2 vision systems / 4 DENSO SCARA + 2 FANUC Large selection of different items Curved transparent / translucid plastic Multiple pattern matching / multiple object views ……
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
21
QuickTime™ and a decompressor are needed to see this picture.
INFN-Laboratori Nazionali del Gran Sasso: BAM, Brick Assembly Machine (2’30’’ movie)
OPERA is a project of > 60 M€ Neutrinos are 'produced' at CERN, Geneva (CH), and travel 730 km to Gran Sasso Labs in Central Italy The detector is composed of 210.000 bricks made out of Pb sheet and photografic emulsion films (Fuji Film) A true production line installed in the Gran Sasso tunnel: 10 assembly robots + 1 transfer robot IL has contributed with the retrofitting of 5 vision systems to achieve / ensure alignment NIDay, Milan, 25-02-09
Tight integration of vision & robotics
22
QuickTime™ and a decompressor are needed to see this picture.
A quick intro to 3D imaging
3D imaging has a great potential in robotics applications Available technologies: laser scanning, stereo-imaging, time of flight ImagingLab is currently working on 3D and has recently released a LabVIEW wrapper for the Ranger series of SICK-IVP cameras Example: random bin-picking (as opposed to palletizing)
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
23
QuickTime™ and a decompressor are needed to see this picture.
A LabVIEW library dedicated to robotics … NIDay, Milan, 25-02-09
Tight integration of vision & robotics
24
QuickTime™ and a decompressor are needed to see this picture.
Why integrators need to use different brands of robot?
Integrators / end user are often faced with the need to ‘mix & match’ different robot brands Based on offer range and characteristics Choice imposed by end customer
Standards exist for cameras, at least at communication / driver level: (old analogue standards at signal level, CCIR, RS170, etc.), IEEE 1394 Firewire, CameraLink, GigE, … Robots of different brands do not share a common programming language Functionally all robots are similar (performances, reliability,etc. not withstanding) An ‘abstraction layer’ can be built in LabVIEW
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
25
QuickTime™ and a decompressor are needed to see this picture.
Three different text-based programming examples Kuka code example Denso code example
1
INI
2
PTP HOME Vel= 100 % DEFAULT
3
GRIPPER_OPEN () ; Open the gripper
4
PTP P_PICK_UP Vel= 100 % P_PICK_UP Tool[1] Base[0] ; Move 100 mm above the pick position.
5
LIN_REL {Z -100} ; Move to the pick position.
6
GRIPPER_CLOSE () ; Close the gripper APPROACH P,P_PICK,100, S=100
7
LIN_REL {Z +100} ; Move 100 mm MOVE above the pick position. L, P_PICK, S=50
'!TITLE "Test Pick&Place"
Mitsubishi code example
PROGRAM TEST_PICKPLACE
10 SERVO ON
TAKEARM
20 OVRD 100
MOTOR ON
30 HOPEN 1
CALL GRIPPER OPEN
position.
' Open the gripper.
40 DLY 0.5 ' Open the gripper ' Move50 100 mm P_PICK,100 above the pick position. MOV 60 OVRD 50 ' Move to the pick
8
PTP P_PLACE_UP Vel= 100 % P_PLACE_UP Tool[1] Base[0] ; Move 100 mm aboveP_PICK the place position. 70 MVS
9
LIN_REL {Z -100} ; Move to the place position. DEPART P,100, S=100
10
GRIPPER OPEN () ; Open the gripper
11
LIN_REL {Z +100} ; Move 100 mm above the place position.
12
PTP HOME Vel= 100 % DEFAULT
CALL GRIPPER_CLOSE the pick position.
' Move 100 mm above the pick position.
' Close the gripper
80 HCLOSE 1 ' Move 100 mm above
' Move to the pick position. ' Close the gripper.
90 DLY 0.5
APPROACH P,P_PLACE,100, S=100 ' Move 100 mm above the place position. 100 OVRD 100 MOVE L, P_PLACE, S=50 position.
' Move to the place 110 MVS,100
' Move 100 mm above the pick position.
CALL GRIPPER_OPEN
120 MOV P_PLACE,100 ' Open the gripper
' Move 100 mm above the place position.
DEPART P,100, S=100 the place position.
130 OVRD 50' Move 100 mm above 140 MVS P_PLACE
' Move to the place position.
150 HOPEN 1
' Open the gripper.
MOTOR OFF GIVEARM 160 DLY 0.5 END 170 OVRD 100
NIDay, Milan, 25-02-09
180 SERVO OFF Tight integration of vision & robotics
26
QuickTime™ and a decompressor are needed to see this picture.
Building an ‘abstraction layer’: how to handle different brands of robots
An ‘action’ or command is essentially identical for different brands of robots Different robot manufacturers use different comunication techniques / different protocols / different languages High-level commands common to all brands / specific 'plug-ins' to handle different protocols High level LabVIEW robot protocol
Command
Denso Protocol
Mitsubishi Protocol
Kuka Protocol
Staubli Protocol
…
XYZ Protocol
Robot Controller NIDay, Milan, 25-02-09
Tight integration of vision & robotics
27
QuickTime™ and a decompressor are needed to see this picture.
Key points of LabVIEW integration: a library and tools for high-level robotics programming
Real value is the combination of robotics and vision into a single LabVIEW development environment:
One single development platform for robotics, vision, feeding / loading devices, generic data acquisition & measurements, HMI, standard industrial protocols, web services, … Multiplatform: Windows, Linux, MacOS, RT OS, FPGA, … Better performances Faster development Easier interface Add-on high-level tools (image calibration, automatic robotics-vision calibration, …) In complex applications the machine vision is 'master‘
Remove complexity of syntax and specific robotics language / command with LabVIEW
For different families of robots the LV-based library establishes a direct communication with the controller NIDay, Milan, 25-02-09
Tight integration of vision & robotics
28
QuickTime™ and a decompressor are needed to see this picture.
Robotics, vision, feeder and HMI in LabVIEW
A simplified diagram of a whole application
Select vision recognition algorithm
Type of feeder NIDay, Milan, 25-02-09
Type of robot
Enable MV quality check
Pick & place coordinates
Select vision-based feeder management Tight integration of vision & robotics
29
QuickTime™ and a decompressor are needed to see this picture.
An example of 'abstraction': the ‘move to’ VI
What is hidden behind a simple ‘Move to position’ VI?
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
30
QuickTime™ and a decompressor are needed to see this picture.
A robotics library in LabVIEW for ‘all’ robots
Robots of different brands do not share a common programming language
Integrators / end user are often faced with the need to ‘mix & match’ different robot brands
An ‘abstraction layer’ can be built in LabVIEW
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
31
QuickTime™ and a decompressor are needed to see this picture.
Today’s library and future steps / 1
IL robotics library is migrating from an ‘internal product’ only:
The library is ‘external’ to the controller:
No interaction with the inverse kinematics software / firmware
There are many future attractive issues:
Further work might be required to make it available to other integrators
A future generation of controllers exploiting some of the RT technologies available within NI Dual control loops capable of switching from encoder feedback to vision feedback A LabVIEW-based ‘cross-compiler’ (like today's LV to FPGA) for different robot controllers
Warning: the inherent complexity of ‘future’ steps increases rapidly when compared to today’s library
Requires tighter collaboration with robot manufacturers
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
32
QuickTime™ and a decompressor are needed to see this picture.
Today’s library and future steps … /2
Availability of the library to other integrators / end users Some work is in progress to make the library available outside ImagingLab (few ‘test’ users in Italy, UK, Germany) Pricing, documentation, distribution, support, are some of the issues we are addressing There is an on-going discussion with NI Corporate and major robot manufacturers
First ‘Denso-flavoured’ library fully documented Installer
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
33
QuickTime™ and a decompressor are needed to see this picture.
Interested in our library? Please contact us … Interested in a next seminar dedicated to technical details of integrated vision and robotics programming in LabVIEW? Please contact us …
[email protected] [email protected]
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
34
QuickTime™ and a decompressor are needed to see this picture.
Questions & answers …
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
35
Tight Integration of Machine Vision & Robotics with LabVIEW in flexible manufacturing, automated assembly, packaging and testing Ignazio E M Piacentini [email protected] www.imaginglab.it NIDay 2009, Milan, 25th February 2009
QuickTime™ and a decompressor are needed to see this picture.
Agenda
The machine vision and robotics market: few numbers What is Flexible Manufacturing? Vision and robotics in flexible manufacturing: an example Basic concepts and requirements Few application examples Standards in Machine Vision / lack of standards in robotics Direct communication from LabVIEW to the robots: introducing a robotics library / a robotics toolkit in LabVIEW Questions and answers
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
2
QuickTime™ and a decompressor are needed to see this picture.
Machine Vision and Robotics is a growing mkt
There are plenty of new areas for robots, especially low payload, high accuracy ones Today only 5-10 % of robotics apps make use of MV, but these number is destined to grow: 35% by 2012 ??
Estimated yearly supply of industrial robots in Europe by countries 2005 - 2006 Germany Italy France Spain BENELUX
2006 2005
Central/Eastern Europe
100.000 robots deployed annually around the world 4000 – 6000 robots are deployed in Italy
NIDay, Milan, 25-02-09
United Kingdom Sweden Austria Switzerland Denmark Turkey all other countries
0
2.000
4.000
6.000 units
Tight integration of vision & robotics
8.000
10.000
12.000 Source:World Robotics 2007
3
QuickTime™ and a decompressor are needed to see this picture.
Italy: 'meccanica e macchine' : the weight of Italy in Europe
The weight of Italy in Europe: GDP (GEP?), manufacturing industry, ‘machines’ production Italy ranks at the 4th place, but machine manufacturing is 2nd after Germany
Italians are pretty good at building and exporting ‘macchine speciali’
Weight of Italy in Europe
Germany
UK
France
Italy
Spain
Other EU members
% of Gross European Product
20.0%
16.4%
15.4%
12.8%
8.5%
26.9%
Manufacturing industry
26.0%
11.3%
15.1%
14.2%
7.5%
25.9%
Machine manufacturing
33.4%
9.4%
11.1%
19.5%
5.0%
21.6%
Source: Federmacchine / Eurostat NIDay, Milan, 25-02-09
Tight integration of vision & robotics
4
QuickTime™ and a decompressor are needed to see this picture.
'Macchine speciali', vision, robotics, automation and LabVIEW Just one LabVIEW - NI platform for:
Machine Vision with the Vision Developer Module Robot programming with the ImagingLab robotics library Remote I/O and machine automation with C-RIO / PACs / PLCs HMI and remote machine monitoring
“ LabVIEW Everywhere ! ” NIDay, Milan, 25-02-09
Tight integration of vision & robotics
5
QuickTime™ and a decompressor are needed to see this picture.
Machine vision and robotics: tight integration
Robot application without vision usually require simple programming Very simple applications can be programmed directly from the ‘teaching pendant’ When vision is present, programming gets more complex (dynamic missions / dynamic programming) In machine vision + robotics application vision should not be considered an ‘add-on’, but designed / programmed as a whole unit and possibly in a single programming environment
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
6
QuickTime™ and a decompressor are needed to see this picture.
Why and what is 'flexible manufacturing'? 1: market changes
Market and economics changes have a deep effect on engineering Production methods and requirements have changed a lot in recent years:
Flexibility vs production speed
Smaller production lots Faster time to market Lowering production costs to respond to agressive new competitors (China, India mkts) Cycle time: what does it really means? Design for speed Design for flexibility Design for ease of use Design for low cost of ownership
Pictures courtesy of Automatismi Brazzale and Cabur NIDay, Milan, 25-02-09
Tight integration of vision & robotics
7
QuickTime™ and a decompressor are needed to see this picture.
Why and what is 'flexible manufacturing'? 2: robotics
Assembly machines need to be able to produce different objects / products:
Flexible automated assembly:
At least in the same class of products No re-tooling required Fast production lot change with limited or no operator intervention Used by non skilled personnel Reconfigurable / reusable for new products In-process quality control (as opposed to end-ofline quality control) Loading / feeding components Putting the components together
Flexible and reconfigurable ‘mechanics’ for handling and assembly Highly modular The answer is: more robotics NIDay, Milan, 25-02-09
Tight integration of vision & robotics
8
QuickTime™ and a decompressor are needed to see this picture.
Why and what is 'flexible manufacturing'? 3: machine vision
The robots need to perform more complex and ‘dynamic’ (i.e.) work cycles Robots need to see! Vision is used to: Locate the parts and send info to the robot Perform quality inspection of single components Verify assembly correctness
In-process vs end of line quality control:
Discards only defective component Prevents machine malfunctioning / jamming
The answer is: more vision
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
9
QuickTime™ and a decompressor are needed to see this picture.
Why and what is 'flexible manufacturing'? 4: better integrated software
Machines need to be reconfigurable by the end user Vision and robotics sw needs to have simple user interface The integration of robotics and vision must take care of calibration procedures, machine commissioning and start-up tools, etc The answer is: high level software libraries and tools specific to vision and robotics integration
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
10
QuickTime™ and a decompressor are needed to see this picture.
Flexible manufacturing / flexible feeding: the feeding station
Conventional vibrating bowl-feeder are adjusted for a specific component: lack of flexibility!! Flex feeders: computer-controlled vibration amplitude, direction and frequency Machine vision identifies the components Robots pick and place the part LabVIEW offers one single software platform Advanced tools for auto-calibration Advanced tools for system configuration Quality control of the selected parts NIDay, Milan, 25-02-09
Tight integration of vision & robotics
11
QuickTime™ and a decompressor are needed to see this picture.
Flexible assembly on the factory floor
Co-design and collaboration is a key to success (ImagingLab, Automatismi Brazzale, Cabur) From the early prototype to the deployed machine A short video-clip …
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
12
QuickTime™ and a decompressor are needed to see this picture.
Customers and applications Our customers are small / medium companies, often with strong mechanical background …or large companies / institutions From neutrino detectors, to semiconductors, to packaging, to cosmetics, … Automated assembly, testing, packaging, … Some of the most interesting applications are covered by NDAs and cannot be shown in this presentation We will see a limited number of video-clips All projects are based on LabVIEW + IMAQ + ImagingLab robotics library The robotics library for LabVIEW has been deployed on approx 40 different robotics / vision installations
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
13
QuickTime™ and a decompressor are needed to see this picture.
The essence of vision-robotics integration: the equivalent of LabVIEW 3-icon demo
Acquire the image Extract the information Download info to robot & execute
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
14
QuickTime™ and a decompressor are needed to see this picture.
Machine vision, GPM, and ….
We will not enter into the details of machine vision but .. Geometric and conventional pattern matching need to be complemented by other checks Parts may have different orientations / views It is necessary to discriminate recognizable / pickable / conformal (QC) parts
Some inherent complexity can be removed with a ‘smart’ user interface The end user must be able to set pass / fail criteria, vision and robotics origin and zeros Auto-calibration tools are a must NIDay, Milan, 25-02-09
Tight integration of vision & robotics
15
QuickTime™ and a decompressor are needed to see this picture.
The vision platform: from panel PC to Smart Cameras
The hardware selection is dictated by the application needs:
Conventional PC Industrial Panel PC PXI-based industrial PC (Win-based or RT OS) Compact Vision System (Real Time OS) Smart Camera (Real Time OS)
The software platform remains the same!! See Smart Cam videoclip
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
16
QuickTime™ and a decompressor are needed to see this picture.
A complex application: testing DIMM memory modules @ EEMS (ex Texas Instruments) (1’ 35’’ movie)
NIDay, Milan, 25-02-09
Fully automated functional testing of DIMM memory modules 64 interchangeable motherboards/ 256 memory slots 2 robots with common working space (collision avoidance) Full auto-guide / auto-learn / onboard camera Total testing cycle time compatible with production line Repetitive project
Tight integration of vision & robotics
17
QuickTime™ and a decompressor are needed to see this picture.
‘Smart’ pick and place for the cosmetic packaging
Fast picking from conveyor and accurate placement (< 0.1 mm) Many different sizes and shapes require flexible imaging configuration tool Most repetitive project using inverted Denso SCARA (22 robots, more to come ….) Triggered the development of the direct LabVIEW > b-Cap library Co-design with partner company NIDay, Milan, 25-02-09
Tight integration of vision & robotics
18
QuickTime™ and a decompressor are needed to see this picture.
‘Smart’ pick and place for the cosmetic packaging /2
Adding the brush to face-powder kit An early testing video-clip
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
19
QuickTime™ and a decompressor are needed to see this picture.
EST Projet (Educare per la Scienza e la Tecnologia): teaching vision & robotics at the ‘Leonardo da Vinci’ Science Museum in Milan Un progetto, anzi due, per il Museo della Scienza di Milano
NIDay, Milan, 25-02-09
Two ‘machines’ sponsored by NI, Mitsubishi, ImagingLab Recognising shape and colour, assembling and disassembling Interactive Taking care of an audience of variable age / designing for zero or little maintenance
Tight integration of vision & robotics
20
QuickTime™ and a decompressor are needed to see this picture.
Conveyor tracking in pharmaceutical packaging
Dual line with 2 vision systems / 4 DENSO SCARA + 2 FANUC Large selection of different items Curved transparent / translucid plastic Multiple pattern matching / multiple object views ……
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
21
QuickTime™ and a decompressor are needed to see this picture.
INFN-Laboratori Nazionali del Gran Sasso: BAM, Brick Assembly Machine (2’30’’ movie)
OPERA is a project of > 60 M€ Neutrinos are 'produced' at CERN, Geneva (CH), and travel 730 km to Gran Sasso Labs in Central Italy The detector is composed of 210.000 bricks made out of Pb sheet and photografic emulsion films (Fuji Film) A true production line installed in the Gran Sasso tunnel: 10 assembly robots + 1 transfer robot IL has contributed with the retrofitting of 5 vision systems to achieve / ensure alignment NIDay, Milan, 25-02-09
Tight integration of vision & robotics
22
QuickTime™ and a decompressor are needed to see this picture.
A quick intro to 3D imaging
3D imaging has a great potential in robotics applications Available technologies: laser scanning, stereo-imaging, time of flight ImagingLab is currently working on 3D and has recently released a LabVIEW wrapper for the Ranger series of SICK-IVP cameras Example: random bin-picking (as opposed to palletizing)
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
23
QuickTime™ and a decompressor are needed to see this picture.
A LabVIEW library dedicated to robotics … NIDay, Milan, 25-02-09
Tight integration of vision & robotics
24
QuickTime™ and a decompressor are needed to see this picture.
Why integrators need to use different brands of robot?
Integrators / end user are often faced with the need to ‘mix & match’ different robot brands Based on offer range and characteristics Choice imposed by end customer
Standards exist for cameras, at least at communication / driver level: (old analogue standards at signal level, CCIR, RS170, etc.), IEEE 1394 Firewire, CameraLink, GigE, … Robots of different brands do not share a common programming language Functionally all robots are similar (performances, reliability,etc. not withstanding) An ‘abstraction layer’ can be built in LabVIEW
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
25
QuickTime™ and a decompressor are needed to see this picture.
Three different text-based programming examples Kuka code example Denso code example
1
INI
2
PTP HOME Vel= 100 % DEFAULT
3
GRIPPER_OPEN () ; Open the gripper
4
PTP P_PICK_UP Vel= 100 % P_PICK_UP Tool[1] Base[0] ; Move 100 mm above the pick position.
5
LIN_REL {Z -100} ; Move to the pick position.
6
GRIPPER_CLOSE () ; Close the gripper APPROACH P,P_PICK,100, S=100
7
LIN_REL {Z +100} ; Move 100 mm MOVE above the pick position. L, P_PICK, S=50
'!TITLE "Test Pick&Place"
Mitsubishi code example
PROGRAM TEST_PICKPLACE
10 SERVO ON
TAKEARM
20 OVRD 100
MOTOR ON
30 HOPEN 1
CALL GRIPPER OPEN
position.
' Open the gripper.
40 DLY 0.5 ' Open the gripper ' Move50 100 mm P_PICK,100 above the pick position. MOV 60 OVRD 50 ' Move to the pick
8
PTP P_PLACE_UP Vel= 100 % P_PLACE_UP Tool[1] Base[0] ; Move 100 mm aboveP_PICK the place position. 70 MVS
9
LIN_REL {Z -100} ; Move to the place position. DEPART P,100, S=100
10
GRIPPER OPEN () ; Open the gripper
11
LIN_REL {Z +100} ; Move 100 mm above the place position.
12
PTP HOME Vel= 100 % DEFAULT
CALL GRIPPER_CLOSE the pick position.
' Move 100 mm above the pick position.
' Close the gripper
80 HCLOSE 1 ' Move 100 mm above
' Move to the pick position. ' Close the gripper.
90 DLY 0.5
APPROACH P,P_PLACE,100, S=100 ' Move 100 mm above the place position. 100 OVRD 100 MOVE L, P_PLACE, S=50 position.
' Move to the place 110 MVS,100
' Move 100 mm above the pick position.
CALL GRIPPER_OPEN
120 MOV P_PLACE,100 ' Open the gripper
' Move 100 mm above the place position.
DEPART P,100, S=100 the place position.
130 OVRD 50' Move 100 mm above 140 MVS P_PLACE
' Move to the place position.
150 HOPEN 1
' Open the gripper.
MOTOR OFF GIVEARM 160 DLY 0.5 END 170 OVRD 100
NIDay, Milan, 25-02-09
180 SERVO OFF Tight integration of vision & robotics
26
QuickTime™ and a decompressor are needed to see this picture.
Building an ‘abstraction layer’: how to handle different brands of robots
An ‘action’ or command is essentially identical for different brands of robots Different robot manufacturers use different comunication techniques / different protocols / different languages High-level commands common to all brands / specific 'plug-ins' to handle different protocols High level LabVIEW robot protocol
Command
Denso Protocol
Mitsubishi Protocol
Kuka Protocol
Staubli Protocol
…
XYZ Protocol
Robot Controller NIDay, Milan, 25-02-09
Tight integration of vision & robotics
27
QuickTime™ and a decompressor are needed to see this picture.
Key points of LabVIEW integration: a library and tools for high-level robotics programming
Real value is the combination of robotics and vision into a single LabVIEW development environment:
One single development platform for robotics, vision, feeding / loading devices, generic data acquisition & measurements, HMI, standard industrial protocols, web services, … Multiplatform: Windows, Linux, MacOS, RT OS, FPGA, … Better performances Faster development Easier interface Add-on high-level tools (image calibration, automatic robotics-vision calibration, …) In complex applications the machine vision is 'master‘
Remove complexity of syntax and specific robotics language / command with LabVIEW
For different families of robots the LV-based library establishes a direct communication with the controller NIDay, Milan, 25-02-09
Tight integration of vision & robotics
28
QuickTime™ and a decompressor are needed to see this picture.
Robotics, vision, feeder and HMI in LabVIEW
A simplified diagram of a whole application
Select vision recognition algorithm
Type of feeder NIDay, Milan, 25-02-09
Type of robot
Enable MV quality check
Pick & place coordinates
Select vision-based feeder management Tight integration of vision & robotics
29
QuickTime™ and a decompressor are needed to see this picture.
An example of 'abstraction': the ‘move to’ VI
What is hidden behind a simple ‘Move to position’ VI?
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
30
QuickTime™ and a decompressor are needed to see this picture.
A robotics library in LabVIEW for ‘all’ robots
Robots of different brands do not share a common programming language
Integrators / end user are often faced with the need to ‘mix & match’ different robot brands
An ‘abstraction layer’ can be built in LabVIEW
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
31
QuickTime™ and a decompressor are needed to see this picture.
Today’s library and future steps / 1
IL robotics library is migrating from an ‘internal product’ only:
The library is ‘external’ to the controller:
No interaction with the inverse kinematics software / firmware
There are many future attractive issues:
Further work might be required to make it available to other integrators
A future generation of controllers exploiting some of the RT technologies available within NI Dual control loops capable of switching from encoder feedback to vision feedback A LabVIEW-based ‘cross-compiler’ (like today's LV to FPGA) for different robot controllers
Warning: the inherent complexity of ‘future’ steps increases rapidly when compared to today’s library
Requires tighter collaboration with robot manufacturers
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
32
QuickTime™ and a decompressor are needed to see this picture.
Today’s library and future steps … /2
Availability of the library to other integrators / end users Some work is in progress to make the library available outside ImagingLab (few ‘test’ users in Italy, UK, Germany) Pricing, documentation, distribution, support, are some of the issues we are addressing There is an on-going discussion with NI Corporate and major robot manufacturers
First ‘Denso-flavoured’ library fully documented Installer
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
33
QuickTime™ and a decompressor are needed to see this picture.
Interested in our library? Please contact us … Interested in a next seminar dedicated to technical details of integrated vision and robotics programming in LabVIEW? Please contact us …
[email protected] [email protected]
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
34
QuickTime™ and a decompressor are needed to see this picture.
Questions & answers …
NIDay, Milan, 25-02-09
Tight integration of vision & robotics
35
Related Documents
Flexible Manufacturing Systems
June 2020 10
Flexible Manufacturing Systems
November 2019 14
Laici Responsabili Per Il Vangelo E Per Il Mondo
November 2019 20
53225-mt----flexible Manufacturing Systems
October 2019 20More Documents from "SRINIVASA RAO GANTA"
Nidays 2009 Keynote - Parte I
December 2019 18
Nidays 2009 Keynote - Parte Ii
December 2019 23
