10-4

2002/8/5

Content Mail Delivery Restrictions ∼ SPAM countermeasures ∼

n

Illegal Relay (platform use) prevention

n

SPAM elimination

1

Request for Receipt/Delivery Restrictions (1)

Mail System Issues in the Past

n

n

Principle goal was to deliver mail n Liberal to relaying and source routing u For F

F

n

outside organization to outside organization ( platform platform))

Designate any host along the way as gateway

F

small bandwidth

F

n

è Waste

of time(transfer time, organizing mail)

n

organization only

Relay of mail unrelated to one’ one ’s organization

students, partpart -timers, temporary staff

inside organization

Issue of mailbox size u Overflow

of SPAM, so important mails can’ can’t be

received F

4

Illegal Relay(Platform Use) Rejection

Delivery destination restrictions by policy u within

Third--Party Mail Relay Third

è Stress on CPU, network, disk 3

Request for Receipt/Delivery Restrictions (2) n

Unsolicited Commercial Email (UCE)

Unsolicited mail relay u From

for unstable/incomplete path control

MX backback-up by other organization

Bombing

u Spam

network interconnection

u Support F

Receipt of unsolicited mail u Mail

Designate gateway explicitly user%domain@gateway @gateway:user@domain

u Support F

2

SMTP

Kind of Dos attack

u Disk

administration issue

“ leave on server” server” with POP, IMAP F people who don’ don’ t read mail

MTA

F

5

MUA

MTA or MUA

6

• expiration processing

1

2002/8/5

Role of SPAM Platform

Factors for Delivery Restriction

n

n

From 1 piece of mail with x numbers of destination addresses, x reproductions are made è Influence on platform host

SMTP connection source/destination host’’s host u IP

u Sender

line fees, CPU resources shouldered u Suspicion of sending SPAM, dealing with claims u Tarnish image of organization u Be put on blacklist, mail won’ won’ t arrive

n

u Reject

n n

7

Header content Body content

like sealed letter

n

Header address

When together u Sent

u Enclosed

letter sender/recipient u Generally cannot be rewritten during delivery n

n

Otherwise u Sending

u Sender/recipient

as indicated on front be rewritten during delivery if needed u Remains in UNIX From and Return Return--Path: after receipt

F

u Can

F

from mailing list

Envelope sender/recipient rewritten

u After

processing forward/after processing alias

Envelope sender (recipient too?) rewritten

9

Note that this only remains

10

Header and Envelope at SMTP HELO mx1.s.domain 250 post.r.domain Hello mx1.s.domain MAIL FROM:<[email protected]> 250 sender ok RCPT TO: 250 recipient ok DATA 354 Enter mail, end with "." on a line by itself From: [email protected] To: list @s.domain Subject: Newsletter ← empty line(no empty space either) Tutorial Announcement [Rest omitted ] . 250 Message accepted for delivery

by user (to individual, right after drop off)

User designates header only F Envelope copied from header F

Envelope address

F

8

Header and Envelope Relationship

Header and Envelope n

mail address mail address

u recipient

explicit transfer (source routing) unneeded transfer of external to external

*Looks

Envelope’’s Envelope u sender

è Countermeasure u Reject

address name (by DNS reverse lookname  look- up)

u domain

When to Restrict During SMTP n

When SMTP connects n Response during SMTP session

Envelope sender Envelope recipient

u HELO/EHLO

host name FROM: <envelope sender address> address > u RCPT TO: <envelope recipient address> address> u DATA (header, body content) u MAIL

header body

This part reaches recipient n

After SMTP receipt completion/before sending è to

11

F

reduce burden, better to reject during SMTP generating of error mail becomes sender side’ side’ s job

12

2

2002/8/5

Which Patterns Should be Restricted

Transfer Restrictions and Mailing List, Transfer Configurations n

n

From outside organization to outside organization u Allow

Participation in external ＭＬ Transfer from external account u FROM

「From:inside organization」 organization」?

Ù

n

What about 「From:inside organization」 organization」 from outside organization? organization? n What about 「 From:outside organization」 organization」 from inside organization? organization? n

External participants of internal ＭＬ Transfer to external u FROM

u

ML that rewrites envelope sender is OK

Relay Conformation Classification

14

Relay Judgement Generalization

From internal host (can trust inside organization?) u FROM

inside TO outside outside TO outside u FROM inside TO inside u FROM outside TO inside u FROM

n

outside organization comes from inside

Allow connection from internal host Ù Require sender rewrite/prohibit transfer? Ù

13

n

inside organization comes from outside

If destination is within organization, allowed?

n

OK OK? (likely internal ML) OK? (prefer direct sending) sending) OK?

f (h, s, r) = OK, NO

From external host

h - SMTP connection source host’ host’ s address s - SMTP envelope sender r - SMTP envelope recipient

u FROM

inside TO outside NO? (use ISP ) Beware! u FROM outside TO outside NO (illegal relay) u FROM u FROM

inside TO inside outside TO inside

OK OK

Judgement functions with 3 parameters

(outside ML, might reject) reject) 15

16

Pros and Cons of Allowing “From Inside” From Outside Organization

About “To Outside ” From Outside Organization

n

n

Want to Dial up ISP and send mail with From as my organization è Can deceive address, become platform u Depending

on ISP mail server, can’ can’t send mail from inside except with granted address as From

è Existence

of ML that won’ won’t rewrite envelope

Beware when accommodating backback-up MX for other organizations! u Problems

will be discovered when primary MX is down u Confirm with tests such as http://www.wide.ad.jp/~motonori/mtachecker.html http:// www.wide.ad.jp/~motonori/mtachecker.html  

sender u Returned

mail would be rejected 17

18

3

2002/8/5

Actual Configurations n

sendmail

n

qmail

With sendmail n

Flexible rejection configurations possible from sendmail 8.8/8.9 u Need

to retrieve rejection functions from sendmail.cf

original sendmail attached m4 version generating tool F After CFCF-3.6W F

u Reject

configuration is built in as default of sendmail.cf for sendmail 8.9 ( m4, CFCF- 3.7W)

n

For configuration, can simply add to conventional sendmail.cf u For

19

n

Rejection after checking header content

n

Pattern match function with regular expression

n

Differentiation between “doesn doesn’’t exist” exist” and “couldn couldn’’t find” find” in DNS

Selection of connection source host

u Can Can’’ t

u check_relay

n

Checking of envelope sender address Checking of envelope recipient address

n

Checking of address after receipt

u check_rcpt

u Either

n

u check_compat 21

n

u Possibility

F

u to F

u to F

of freely being designated by MX MX... ...

n

LOCAL_HOST_＊ LOCAL_HOST_ ＊

n

CLIENT_＊ CLIENT_ ＊

n

ROAM_＊ ROAM_ ＊

uf

(trusts inside organization)

${client_addr ${ client_addr}} is outside organization u to

classify as error immediately or save in queue

22

Parameter for Using CF (cont.)

${client_addr}} (connection source host IP ${client_addr address) is inside organization u OK

addresses that couldn’ couldn ’ t exist

Auto recognition of backback-up MX designation

Check Procedure at check_rcpt n

judge combination of multiple headers...

u Expresses

u check_mail

n

20

What’s Possible with sendmail 8.9

What can be Done with sendmail 8.8 n

CF- 3.7W, possible to generate added parts only CF-

uf

own host OK

uf

inside organization

( src_host,,＊,＊) = OK/NO (src_host ( src_host,, from_domain ,＊) = OK/NO (src_host ( src_host,, from_user (src_host from_user,,＊) = OK/NO

OK

outside organization NO (return error during SMTP)

23

24

4

2002/8/5

Class Matching of Partial Network Address (3.1W Function)

Parameter for Using CF (cont’d) n

uf

(*, from_domain ,＊) = OK/NO F

n

n

ALLOW_RELAY_FROM

n

If disguised as from_domain could relay mail from anywhere, so it’ it’ s risky

n

ALLOW_RELAY_TO uf

n

( ＊,＊, to_domain (＊ to_domain)) = OK/NO F

If accommodating lower MX don’’t forget definition don 25

For qmail (cont.) n n

n

Judge by pattern matching of line of characters in sendmail IP address matching is judged in octet units In the CIDR era, need more detailed judging With 8.9.1+3.1W patch, netmask notation possible C{Network} 200.3.4.64/27 C{Network} _ MASKED_ADDRESS_MATCH_ u In this case, 200.3.4.64 - 200.3.4.91 matches For map, use maskedaddr map

For qmail (cont ’d) n

Reject relay in default To allow relay

Mail transfer from local client u Configure

RELAYCLIENT when launching qmail-- smtpd qmail

u Add

addresses allowed transfer in /var/qmail/control/rcphosts F

F

launch from tcp_wrapper and tcpserver (ucspi ucspi--tcp tcp))

u By

using tcpserver tcpserver,, it ’ s possible to control relay after looking at client address

Remember to include addresses provided to Lower MX

F F

IDENT information can also be used rules can go into effect immediately with tcprules command

27

To Confirm Operation after Configuration

n

Use outside organization account n Use WWW page for testing

Authentication of mail sent from outside organization u Simultaneous

u http://maps.vix.com/tsi/ar http://maps.vix.com/tsi/ar-- test.html

use of POP authentication

1) Access POP server 2) Register access base address in database 3) Search database during SMTP 4) If listed in database, access is allowed 5) After certain period of time, deleted from database

u http:// http://www.wide.ad.jp/~motonori/mtachecker.html www.wide.ad.jp/~motonori/mtachecker.html

Be careful to avoid rejection of valid mails! u Difficult

28

Authentication of Mail Sender

n

l

26

to contact host with configuration mistake

u If 29

DB is made with makemap in sendmail, rebooting of sendmail not necessary

30

5

2002/8/5

About SPAM

SPAM Prevention n

n

Name of food product by Hormel Foods Corporation n Receipt of unsolicited mail u Mail

u Submissions

to NetNews lists for mailing lists u Address listed on Web (information collecting robot) u Participant

Bombing

n

u Spam

cost is extremely cheap!

SPAM prevention measures u Altering

Unsolicited Commercial Email (UCE) F Unsolicited Bulk Email (UBE) F Sent indiscriminately regardless of recipients’ recipients’ interests F Some are illegal F

u Promotion

How does your mail address become known?

address such as [email protected]

Mechanical automatic reply function can’ can’ t be used F Troublesome to reply F Tricky for beginners F

u Don Don’’t

respond to delete request from SPAM list?

31

32

SPAM Filtering

Where SPAM Comes From n n

Hosts that allow illegal relay n Directly without relay

What to go by u Sender

mail address (regular address) domain name u Sender host network address u Sender

u Because

it ’ s not illegal relay, there is no perfect it’ countermeasure

F

with or without DNS reverse looklook-up configuration?

u Header F

content

detect typical SPAM characteristics

u Body

content?

33

34

Mailing List and SPAM Countermeasure

Problems with Filtering n

Mail from defined host won’ won’t arrive

n

4 Recipient

can categorize with a mark instead of completely rejecting 4 Measure needed to allow mail that ’ s not relay (from that organization)

u Reject

mail other than from registered members

acquire submission, archive search F acquire member list F

n

If unknown platform is used, effect is weakened n Maintenance of blacklist is troublesome Ù Refer

Standard (?) functions of recent mailing list servers

u Reject

if ML address is missing from header destination

n

to MAPS RBL, etc. 35

Counter with server program separate from MTA u Program

function developments progressing?

36

6

2002/8/5

SPAM Host Databases

MAPS RBL n

n

MAPS RBL n ORBS n DUL

Mail Abuse Protection System Realtime Blackhole List u http:// http://maps.vix.com/rbl maps.vix.com/rbl//

n

Rejected if applicable A record for 4.3.2.1.rbl.maps.vix.com 4.3.2.1. rbl.maps.vix.comin in DNS exists u When u Test

connection request is from IP address 1.2.3.4 address for DNS reference： reference：127.0.0.2

2.0.0.127.rbl.maps.vix.com 2.0.0.127. rbl.maps.vix.com n

Also done by BGP

37

38

ORBS

DUL

n

n

Open Relay Blocking System http://www.dorkslayers.com http:// www.dorkslayers.com/orbs/ /orbs/ n Reference method of list is same as MAPS RBL

ORCA DialDial-up User List u For

n

rejecting direct mails from dialdial- up users

n

http://www.orac.bc.ca/dul// http://www.orac.bc.ca/dul n Method of reference is same as MAPS RBL

u Confirm

existence of A record applicable to 4.3.2.1.orbs.dorkslayers.com

u Confirm

existence of A record applicable to 4.3.2.1.dul.orac.bc.ca u Zone transferring also possible 39

40

Registering Regular spammers Locally n

sendmail (CF)

n

qmail

Check Regarding Sender Address ∼ To Reduce Junk Mail∼ + For

SPAM which cannot be replied to, false sender addresses are common u When @domain is omitted u When format is user@host ( not FQDN) u When not registered with DNS(can DNS(can’’ t be replied to)

u SPAM_LIST*

u control/ control/badmailfrom badmailfrom

F

If it just couldn’ couldn’t be found, not classed as receipt error (temporary reject)

u When

Maintenance is done manually

F

obviously forgery

User part is all numbers, too long • Check with regular expression

41

n

But if address is fake, impossible to distinguish

42

7

2002/8/5

Rejection from Regular Expression

Reject from Header Content (After 8.9)

(After sendmail 8.9) Kcheckaddress regex -a@MATCH ^([0^([0 -9]+<@(aol|msn) 9]+<@(aol|msn)¥ ¥.com|[0 .com|[0-9][^<]* <@juno¥ <@juno ¥.com|.{10}[^<]+<@ .com|.{10}[^<]+<@aol aol¥ ¥.com) .com)¥ ¥.?>

HTo: $> CheckTo HTo: SCheckTo R friend@$* $#error $: "553 Header error"

R $+

$: $(checkaddress $(checkaddress $1 $)

R @MATCH

$#error $: "553 Header error"

HMessage-Id: $> CheckMessageId HMessageSCheckMessageId R < $+ @ $+ > $@ OK R $* $#error $: "553 Header error" n

Opening Contact Point for Complaints n

44

Additional SPAM Countermeasures n

What to do with Envelope senders that are <>

n

If sender were disguised with real address... address ...

n

In the end, can only counter at individual level?

Creation of abuse@domain

u Mail

u ( RFC2142

Mailbox Names for Common Services, Roles and Functions)

n

Implemenation of SpamCan also available http://consult.ml.org/~timb/spamcan http:// consult.ml.org/~timb/spamcan//

43

[email protected] u Network

Abuse Clearinghouse (http://www.abuse.net)

from Mailer_daemon

u Allow u Allow F

45

those with your address in the header only mailing lists you’ you’ re participating in

use procmail etc. 46

Additional Technical Countermeasures n

MTA that can check mail content also

n

Wide usage of electronic signatures

n

Authentication between MTA?

u Firewall

u Throw

products, etc.

out those with no signature

47

8