Dpa Annual Conference 2008: Web Crawling: What’s Legal And What’s Not

DPA ANNUAL CONFERENCE 2008 Web Crawling: What’s Legal and What’s Not Laurie Kaye Laurence Kaye Solicitors Legal solutions for the digital age

June 7 2005

DPA Conference 2005

Agenda 1. “Web crawling” 2. Is this really about the law? 3. Legal propositions 4. Conclusions / Predictions

June 7 2005

DPA Conference 2005 Laurence Kaye www.laurencekaye.com

“Web Crawling” 

“Web crawling, Screen scraping, Web scraping, Web harvesting”



Crawl – request copy of page



Follow links to other pages / resources



Store data / page



Present resource to end user (in full, snippet, thumbnail, etc)

June 7Laurence 2005 Kaye

www.laurencekaye.com

DPA Conference 2005

Is this really about the law?

Is this really about the law?

June 7 2005



Yes it is!



Ryanair v. Vtours (Germany)



Google v. Copiepresse

Laurence Kaye www.laurencekaye.com DPA Conference 2005

Is this really about the law?

Is this really about the law?

June 7 2005



“No it isn’t”



Web > Wild West of Cyberspace > ‘Web norms’



‘Web Norms’

Laurence Kaye www.laurencekaye.com DPA Conference 2005

‘Web Norms’ as starting point 

“I believe that we should be able to decide who is allowed to use our

content – permissions are ours to give not other people’s to take.” 

“I believe that search engines are essential to our business, in

enabling customers to find our content.” 

“I am concerned that third parties may seek to monetize exploitation

of our

June 7Laurence 2005 Kaye www.laurencekaye.com DPA Conference 2005

digital content without our permission.”

‘Web Norms’ as starting point

‘Web Norms’ as starting point 

“I believe that a significant element of our revenues will be derived

from digital exploitation of our content by 2012.” 

“I recognise that an ability to express permissions in a machine-

readable form is an essential element of infrastructure for managing relationships in the Laurence www.laurencekaye.com June 7online 2005 Kaye supply chain.” DPA Conference 2005

Legal Propositions 1. There are rights in website content. 2. Bots, crawlers, etc. are doing certain things that can be made subject to content owner’s permission 3. Expression of permissions must be automated, ‘machine to machine’

Laurence Kaye www.laurencekaye.com

June 7 2005

DPA Conference 2005

1st Proposition: Rights in Website Content

Content and Database Right S

T

C

O

R

U

C

T

U

R

E

I G

H

T

C

O

N

T

E

N

T

• P

Y

R

S ( I n

" w

h

o le

" R

o r

s u

b

e s t r ic t e d E

June 7 2005

x c e p

t io n

U

I G E N v e s t m e n

E t

R in

I CS O P Y R I G c o n t e n t )

s t a n " t s i au l b p s a t ar t n " t i a l

p "a w r t h " o l e

AN c o t s " "e x t r a c t i o n

o r

s

Laurence Kaye www.laurencekaye.com

"

T

C

s u

b

s t a

" "r Re - e u s t tl i r l i cz ta e t di o nA " c E

DPA Conference 2005

o r

H

x c e p

t io n

s

O

2nd Proposition: Bots, etc., need “permission”

Crawlers, etc., need permission?  Screen scraping infringes copyright + database right? (Ryanair v. Vtours, Germany, 2008)  Screen scraping breaches online terms and conditions? (Ryanair v. Vtours, Germany, 2008)  “Extraction” and “reutilisation” are broad concepts (Advocate General ruling following Directmedia Publishing v. Albert-Ludwigs-Universitat, Laurence Kaye www.laurencekaye.com June 7 2005 DPA Conference 2005 Germany,

2nd Proposition: Bots, etc., need “permission”

Crawlers, etc., need permission? 

Googlebot’s extraction of newspaper content + link to cache copy = copyright infringement (Copiepresse v. Google, Belgium, 2007)



Googlebot’s indexing and displaying of snippets of news articles on

its Google News service infringed copyright (Copiepresse v. Google, Belgium, 2007) 

Google couldn’t rely on ‘caching’ exemption in E-Commerce

Directive. Laurence Kaye June 7 2005

www.laurencekaye.com

DPA Conference 2005

3rd Proposition: Permissions must be automated

“Machine Readable Permissions” 

Expressing access and use policies in a machine readable

format 

Crawler operators must implement!



Variety of permissions



With / without enforcement

June 7 2005Laurence Kaye www.laurencekaye.com DPA Conference 2005

Conclusions / Predictions

1. Online business models are the starting point 2. “I believe that a significant element of our revenues will be derived from digital exploitation of our content by 2012” 3. ACAP or ACAP+ will become the norm 4. Permissions will need to accommodate accepted norms for legal exceptions 5. There’s a need for all the players to collaborate to establish norms and make it all work for benefit of all Laurence Kaye www.laurencekaye.com June 7 2005

DPA Conference 2005

Laurence Kaye Solicitors Legal solutions for the digital age

Laurie Kaye [email protected]

Mailin Bala [email protected]

Yasmin Joomraty [email protected] June 7 2005

T: 01923 352 117 Web: www.laurencekaye.com Blog: http://laurencekaye.typepad.com

DPA Conference 2005