Fetchmail
As you already know from by previous postfix article, Any mail going through drunkenmonk.org domain will have my gmail address as sender. Thats good. What about receiving mail from gmail?
In your mind, you are saying “Its damn easy right? configure your evolution/thunderbird to poing to gmail stupid?” right? yeah, thats right. I can configure any mail client to point to gmail and get mails. But, I want my postfix to play a roll.
Postfix is basically a smtp server, it can also act like smtp client. But, it cant work as a imap/pop3 client.
So, Here is the task, I need to get mails from gmail through pop3 and put it into my postfix so that all other clients connected with my postfix can get that mail. Here comes fetchmail.
Fetchmail is a beautiful program which will act like imap/pop3 client (just like evolution/thunderbird) and instead of saving it somewhere, fetchmail will put that mail into your local machine’s mail server. So, all the mail clients connected with your local mail server will get your email from gmail.
Thats enough of theory, lets get down and install fetchmail, first install the package
$ sudo equo install fetchmail $
Now time to configure fetchmail, it uses /etc/fetchmailrc as its only configuration file. But, we need to be careful with it. Because, we are saving the username/password a plain-text, so the permission for /etc/fetchmail should be (0600) for root.
If you start configuring fetchmail by reading ‘man fetchmail’, you will switchoff easily. That man page is such a damn brief and will confuse you easily. But, fetchmail config is really a easy one. It will take just three lines to configure it. Here is those three lines,
set daemon 120 set syslog poll pop.gmail.com proto pop3 user "foo" pass "bar" to "stupid" ssl sslproto "TLS1"
Here, ‘set daemon 120′ will make fetchmail to connect to gmail server every 2 minutes.
‘set syslog’ will make fetchmail to put logs into syslog.
‘poll pop.gmail.com’ instructs fetchmail to connect to pop.gmail.com server.
‘proto pop3′ instructs fetchmail to use pop3 protocol.
‘user “foo” pass “bar” to “stupid”‘ instructs fetchmail to use ‘foo’ as username and ‘bar’ as password to open gmail account in gmail server, fetch those mail from gmail server and put it into local machine’s postfix as a mail for ‘stupid’ user.
‘ssl’ instructs fetchmail to use secure connection. ‘sslproto “TLS1″‘ tells fetchmail to use STARTTLS for sercure connection.
Thats it for configuration stuff, now its time to restart fetchmail
$ sudo eselect rc restart fetchmail $
Now, you can see your mails are fetched from gmail and available to you through ‘mailx’ command.
Dovecot (with mbox)
There are different ways to save a mail in *nix systems. The old baddy is mbox which saves all your email in one single file. The new one is Maildir.
Even though mbox have difficulties, traditional commandline mail clients like ‘mailx’ works with ‘mbox’ style mail files flawlessly. Also postfix which I setup previously by default creates single mbox mail file inside /var/spool/mail/ directory for each user. So, I need a imap+pop3 server which can operate with mbox like mail files. Dovecot is the choice.
First we need to install dovecot. I’m using sabayon, the instruction is specific to sabayon/gentoo,
$ sudo equo install net-mail/dovecot
Its time to setup dovecot. Before we do, we need to create self signed certificates to use it for imaps and pop3s. Lets first create those certificates.
$ cd /etc/ssl/dovecot $ sudo mkdir oldcerts $ sudo mv * oldcerts $ sudo openssl genrsa -out server.key.password -des3 1024 Generating RSA private key, 1024 bit long modulus ..........................................++++++ ..................................................++++++ e is 65537 (0x10001) Enter pass phrase for server.key.password: Verifying - Enter pass phrase for server.key.password: $
We have to give ‘pass phrase’ here, otherwise openssl command will not create key file. But, having pass phrase for key file is not good, because we need to provide this pass phrase everytime dovecot access this file. Here is the step to remove the pass phrase from rsa key file
$ sudo openssl rsa -in server.key.password -out server.key Enter pass phrase for server.key.password: writing RSA key $
Now generate certificate request with the rsa key file
$ sudo openssl req -out server.csr -new -key server.key You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [AU]:IN State or Province Name (full name) [Some-State]:TamilNadu Locality Name (eg, city) []:Chennai Organization Name (eg, company) [Internet Widgits Pty Ltd]:DrunkenMonk Private Limited Organizational Unit Name (eg, section) []:Pattasarayam Generating Unit Common Name (e.g. server FQDN or YOUR name) []:drunkenmonk.org Email Address []:webmaster@drunkenmonk.org Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []: $
We should not give password when creating certificate request. Ok, know time to create self signed certificate.
$ sudo openssl x509 -req -in server.csr -out server.crt -signkey server.key -days 365 Signature ok subject=/C=IN/ST=TamilNadu/L=Chennai/O=DrunkenMonk Private Limited/OU=Pattasarayam Generating Unit/CN=drunkenmonk.org/emailAddress=webmaster@drunkenmonk.org Getting Private key $
Now we have two files /etc/ssl/dovecot/server.key and /etc/ssl/dovecot/server.crt to use it for SSL. Lets configure dovecot now, we need to modify /etc/dovecot/dovecot.conf like this,
protocols = imap pop3 listen = *, ::
Configuration not ends with /etc/dovecot/dovecot.conf, it has different conf files for different purpose inside /etc/dovecot/conf.d/, lets modify one by one, Here is /etc/dovecot/conf.d/10-auth.conf
auth_mechanisms = plain login !include auth-system.conf.ext #!include auth-sql.conf.ext #!include auth-ldap.conf.ext #!include auth-passwdfile.conf.ext #!include auth-checkpassword.conf.ext #!include auth-vpopmail.conf.ext #!include auth-static.conf.ext
If you want to permit only current machine’s users and don’t want to use ldap or other machinisms, then make sure you comment all includes except auth-system.conf.txt.
Here is modifications inside /etc/dovecot/conf.d/10-logging.conf
log_path = syslog syslog_facility = mail
Here is modifications inside /etc/dovecot/conf.d/10-mail.conf
mail_location = mbox:~/.mail:INBOX=/var/mail/%u
Now /etc/dovecot/conf.d/10-ssl.conf
ssl = yes ssl_cert = /etc/ssl/dovecot/server.crt ssl_key = /etc/ssl/dovecot/server.key
Thats all from config stuff, lets restart dovecot.
$ sudo eselect rc restart dovecot
Now, we need to test if it is working, We need to connect with dovecot through imap port with TLS encryption
$ openssl s_client -connect localhost:143 -starttls imap
Above command will show lot of SSL stuffs, and then finally dovecot will say ‘. OK’, we need to start communication with dovecot from there,
. OK Pre-login capabilities listed, post-login capabilities have more. a login mokka somepassword * CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE a OK Logged in b select inbox * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 0 EXISTS * 0 RECENT * OK [UIDVALIDITY 1343790396] UIDs valid * OK [UIDNEXT 320] Predicted next UID * OK [NOMODSEQ] No permanent modsequences b OK [READ-WRITE] Select completed. c list "" * * LIST (\HasNoChildren) "/" "INBOX" c OK List completed.
The above session with dovecot shows that things are ok. You can now configure your Thunderbird/Evolution to use your machine for local emails. Have a nice day!!
Postfix (with smtp.gmail.com as relayhost)
Most of us forget that postfix not only acts like a mail-server, but it can act like a smtp-client (like thunderbird, evolution).
This post is all about configuring postfix to work as mail-server for my local domain (drunkenmonk.org) at the same time forward any outside domain mails to smtp.google.com with my gmail account credentials. If you are curious about drunkenmonk.org, read my previous articles about dnsmasq and virtual home network. These two articles will explain how I setup my home network.
First things first, lets install postfix (Well I’m using sabayon)
$ sudo equo install mail-mta/postfix
BASIC MAIL SERVER
Postfix configuration is pretty simple and straight forward. You have to configure following parameters inside /etc/postfix/main.cf
mydomain = drunkenmonk.org myorigin = $mydomain inet_interfaces = all mydestination = localhost, localhost.$mydomain, $myhostname, $mydomain, mail.$mydomain mynetworks_style = subnet
‘mydomain’ tells postfix that it should serve as mail server for ‘drunkenmonk.org’ domain.
‘myorigin’ tells that postfix should append ‘drunkenmonk.org’ to any incoming mail’s address if the address don’t have domain part. Which means, when you do ‘mail someuser’ from commandline, postfix will append ‘drunkenmonk.org’ so that To address will become ‘someuser@drunkenmonk.org’.
‘inet_interfaces’ tells postfix to listen on all network interfaces.
‘mydestination’ tells postfix that it should be the final destination for ‘localhost’, ‘localhost.drunkenmonk.org’, ‘mokka.drunkenmonk.org’, ‘drunkenmonk.org’ and ‘mail.drunkenmonk.org’
‘mynetworks_style’ tells postfix that my local network is a ‘subnet’ type.
Thats all needed for postfix to get start. We can restart postfix with this configurations to send/receive mails between local users. However we want a little bit more from postfix. Lets configure it to use STARTTLS for its communication with its clients
BASIC MAIL SERVER WITH STARTTLS
If you wonder what is STARTTLS, its a simple method to establish communication in a secure way. Your server application (like Postfix listening on port 25) says “Hey client I have TLS support, If you want, we can communicate each other in TLS secure way within port 25 itself, If you don’t want, we can continue with plain-text, what you say?”, then your client (like Evolution or Thunderbird) will tell “OOh!! you offer TLS, then lets do TLS!!”. So the packets going between server and client will be encrypted using TLS.
Before STARTTLS, people used different port for secure smtp which is smtps(port 465). But nowadays it is changing, the regular smtp port itself been used for both plain-text and SSL/TLS communication.
We have to configure Postfix to talk in SSL/TLS way, for that we need Self Signed SSL certificate. Lets create one,
$ cd /etc/ssl/postfix $ sudo mkdir oldcerts $ sudo mv * oldcerts $ sudo openssl genrsa -out server.key.password -des3 1024 Generating RSA private key, 1024 bit long modulus .............................++++++ ..........................++++++ e is 65537 (0x10001) Enter pass phrase for server.key.password: Verifying - Enter pass phrase for server.key.password: $
Here you have to give a ‘pass phrase’ to generate RSA private key, but having a key with ‘pass phrase’ is asking for trouble. You have to manually type this password everytime postfix access this key. The following command will remove ‘pass phrase’ from a RSA key
$ sudo openssl rsa -in server.key.password -out server.key Enter pass phrase for server.key.password: writing RSA key $
Now you have a RSA private key file server.key without ‘pass phrase’. Its time to create certificate request,
$ sudo openssl req -out server.csr -new -key server.key You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [AU]:IN State or Province Name (full name) [Some-State]:TamilNadu Locality Name (eg, city) []:Chennai Organization Name (eg, company) [Internet Widgits Pty Ltd]:DrunkenMonk Pvt Ltd Organizational Unit Name (eg, section) []: Pattasarayam Generating Unit Common Name (e.g. server FQDN or YOUR name) []:MokkaPandi Email Address []:webmaster@drunkenmonk.org Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []:
Don’t give any password when creating certificate request. Finally create a self signed ssl certificate using following command,
$ sudo openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt Signature ok subject=/C=IN/ST=TamilNadu/L=Chennai/O=DrunkenMonk Pvt Ltd/OU= Pattasarayam Generating Unit/CN=MokkaPandi/emailAddress=webmaster@drunkenmonk.org Getting Private key $
Ok, thats it. You will have two files, one is /etc/ssl/postfix/server.crt and another is /etc/ssl/postfix/server.key, Now configure postfix to use these two for SSL/TLS communication, Add the following lines at the end of /etc/postfix/main.cf
# TLS Support smtpd_tls_cert_file = /etc/ssl/postfix/server.crt smtpd_tls_key_file = /etc/ssl/postfix/server.key smtpd_tls_security_level = may
Finally, start postfix
$ sudo eselect rc restart postfix
Check if postfix can operate with STARTTLS,
$ sudo openssl s_client -connect localhost:25 -starttls smtp
Above command will show the server certificate and all SSL stuffs. After this, whatever you send will be encrypted, now check if it can send mail,
250 DSN EHLO drunkenmonk.org 250-mokka.drunkenmonk.org 250-PIPELINING 250-SIZE 10240000 250-VRFY 250-ETRN 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN mail from: root@drunkenmonk.org 250 2.1.0 Ok rcpt to: webmaster@drunkenmonk.org 250 2.1.5 Ok data 354 End data with . Subject: test mail test mail . 250 2.0.0 Ok: queued as E0E352C0091 quit 221 2.0.0 Bye closed $
Now see if root got a test mail
$ sudo mail >N 8 root@drunkenmonk Tue Aug 7 16:50 14/533 test mail &p From root@drunkenmonk.org Tue Aug 7 16:50:23 2012 X-Original-To: webmaster@drunkenmonk.org Subject: test mail Date: Tue, 7 Aug 2012 16:49:48 +0530 (IST) From: root@drunkenmonk.org test mail &q $
We are not done yet. There is one more part. Lets make postfix to forward mails to gmail.
MAIL FORWARDER
Till now, postfix can send/receive mails within drunkenmonk.org domain. But if someone from your subnet want to send a mail to a person who have mail address outside drunkenmonk.org domain (let say stuid@yahoo.com), then your current postfix will try to contact mail server of yahoo.com. But, yahoo’s mail server will reject your mail because drunkenmonk.org is not a vaild registered domain.
But, If you have a gmail account, you can forward the mail first to smtp.gmail.com with your account credentials so that gmail will send your mail to stupid@yahoo.com with your gmail account credentials. The person who owns stupid@yahoo.com will receive a mail with from address as yours irrespective of whoever send within your local network.
To make postfix to know your credentials, we need to create a credential db with gmail account credentials, For that we need to add following line at the end of /etc/postfix/saslpass file
# run following command to generate hash password db # that can be used inside postfix's main.cf # # command: # postmap hash:saslpass # # #file content format: #remotehost username:password smtp.gmail.com foouser:barpassword
Now run the following command to create saslpass.db
$ cd /etc/postfix $ sudo postmap hash:saslpass
Now you can see saslpass.db created as a Berkeley DB file. We can now use this file inside main.cf to inform postfix about your gmail account credential. Before doing that, please remove /etc/postfix/saslpass file or change its access to 0600 for root. Because your credentials are in plaintext. We need to add following lines at the end of main.cf
# gmail smtp_sasl_auth_enable = yes smtp_sasl_password_maps = hash:/etc/postfix/saslpass smtp_sasl_security_options = noanonymous smtp_tls_security_level = encrypt smtp_tls_mandatory_ciphers = high smtp_tls_verify_cert_match = hostname, nexthop, dot-nexthop relayhost = smtp.gmail.com:587
Here ‘hash:/etc/postfix/saslpass’ actually makes postfix to access /etc/postfix/saslpass.db file. You can remove /etc/postfix/saslpass which have your gmail password as plaintext. It will not affect postfix and postfix doesn’t need it.
Thats it, we are done. Once you restart postfix, It becomes capable of sending mail to outside world but with your gmail account’s credentials. Thanks for reading all the way down to this line. Have a nice day!!
Virtual Home Network (with Dnsmasq , Bridge, TUN/TAP, Qemu)
As I said in my previous article, this post will explain how we can use bridge interface to configure networks inside virtual hosts and much more about virtualization.
First we need to make sure our machine is capable of kvm virtualization, see if you get any output for below command
$ grep -E 'vmx|svm' /proc/cpuinfo
Most modern processers supports Hardware assisted Virtualization. If you don’t get any output for above command, It means you don’t have a processor capable of providing Hardware Assisted Virtualization. Qemu-kvm will not work, but you can use Qemu without kvm or some other virtualization applications like virtualbox etc. Also skip ‘LOADING KERNEL MODULES’ section of this article if you don’t have kvm support.
LOADING KERNEL MODULES
If your processor supports kvm, then we have to load following kernel modules and make sure they load automatically whenever we restart our system. In gentoo/sabayon, we need to modify ‘modules=’ line in /etc/conf.d/modules file as
modules="kvm-intel tun"
Here ‘kvm-intel’ is the kernel module for virtualization and ‘tun’ is the kernel module for ‘TUN/TAP’ devices. For, AMD machines, we need to load ‘kvm-amd’ instead of ‘kvm-intel’. Now we need to load these modules for the current run, Here is the commands which will load ‘kvm-intel’ and ‘tun’ drivers into kernel
$ sudo modprobe kvm-intel tun $
INSTALLING PACKAGES
Next is to install required packages, Here is sabayon command to install required packages
$ sudo equo install app-emulation/qemu-kvm net-misc/bridge-utilities net-dns/dnsmasq sys-apps/usermode-utilities $
‘qemu-kvm’ is for virtualization, ‘bridge-utilities’ is to get ‘brctl’ command, ‘dnsmasq’ is to handle DNS and DHCP requests from guests, ‘usermode-utilities’ is to get ‘tunctl’ command.
CREATING INTERFACES
Its time to create interfaces, first we need to create the bridge
$ sudo brctl addbr br0
Assign IP address to the bridge,
$ sudo ifconfig br0 192.168.2.1 netmask 255.255.255.0 up
Next, create a tap0 interface, following command will create ‘tap0′ pseudo slave interface from /dev/net/tun master interface
$ sudo tunctl -t tap0
Now, we need to hook the tap0 interface with the bridge br0
$ sudo brctl addif br0 tap0
Bring up the tap0
$ sudo ifconfig tap0 up
FIREWALL RULES AND MASQUERADING
If you read my previous article, I have dnsmasq configured to serve DHCP requests coming from br0. Hooking tap0 into br0 will make the guest send DHCP requests via tap0->br0 and my host machine’s dhsmasq process will serve ip address to the guest. We have to make sure iptables dont block BROADCAST packets as well as open the port 53 and 64 so that dnsmasq will get the DHCP and BOOTP packets. Make sure your iptables contains following lines,
$ sudo iptables -t filter -L | grep -E 'BROADCAST|domain|bootp' ACCEPT all -- anywhere anywhere ADDRTYPE match src-type BROADCAST ACCEPT udp -- anywhere anywhere udp dpt:bootps ACCEPT udp -- anywhere anywhere udp dpt:domain ACCEPT tcp -- anywhere anywhere tcp dpt:domainflags: FIN,SYN,RST,ACK/SYN ACCEPT tcp -- anywhere anywhere tcp dpt:bootpsflags: FIN,SYN,RST,ACK/SYN $
Now we need to make sure MASQUERADING is enabled and let your host system forward the packets. Also make these settings permanent so that reboot dont break the settings,
$ sudo iptables -t nat -A POSTROUTING -j MASQUERADE $ sudo sysctl net.ipv4.ip_forward=1 $ sudo /etc/init.d/iptables save $ sudo /etc/init.d/iptables reload $ sudo sed -i 's/^.*net.ipv4.ip_forward.*$/net.ipv4.ip_forward = 1/g' /etc/sysctl.conf
Check these settings are ok,
$ sudo iptables -t nat -L Chain POSTROUTING (policy ACCEPT) target prot opt source destination MASQUERADE all -- anywhere anywhere $ sudo sysctl -a | grep ipv4.ip_forward net.ipv4.ip_forward = 1 $
STARTING QEMU GUEST
Finally, its time to start virtual guest, first we need to create disk image, Here is the command to create a 5GB raw disk image for qemu-kvm
$ sudo qemu-img create -f raw Debian.img 5G $
Now download debian netinstall image for amd64 architecture
$ sudo wget http://ftp.nl.debian.org/debian/dists/squeeze/main/installer-amd64/current/images/netboot/gtk/mini.iso $
Atlast, start the guest with with tap0 networking
$ qemu-kvm -cpu kvm64 -drive file=Debian.img,if=virtio -cdrom mini.iso -boot order=dc -m 512 -soundhw sb16 -name "Debian" -net nic,model=virtio -net tap,ifname="tap0",script=no,downscript=no
Thats it, your qemu guest will automatically get network settings from host machine. You can also verify that dnsmasq served a DHCP request via syslog
$ sudo grep 'dnsmasq-dhcp' /var/log/messages dnsmasq-dhcp[9810]: DHCPDISCOVER(br0) 192.168.2.89 52:54:00:12:34:56 dnsmasq-dhcp[9810]: DHCPOFFER(br0) 192.168.2.89 52:54:00:12:34:56 dnsmasq-dhcp[9810]: DHCPREQUEST(br0) 192.168.2.89 52:54:00:12:34:56 dnsmasq-dhcp[9810]: DHCPACK(br0) 192.168.2.89 52:54:00:12:34:56 virt0 $
Well, if you still reading this article and not get bored, you must be a *nix admin. Have a great day!!
Dnsmasq for Home User
Every BSNL broadband subscribers know that BSNL’s default nameservers are one of the worst in response times. Even if you have connectivity exceeding 1gbps, If your nameservers are not good, then your internet experience will not be good.
So, I decided to use google’s nameservers. I could have configure NetworkManager not to get nameservers through DHCP instead ask it to use google’s nameservers, but I thought of giving ‘dnamasq’ a try.
For those who don’t know what is mean by dnsmasq, it is a dns forwarder which can handle DNS, DHCP and BOOTP requests. It means, it can act like bind, dhcpd and tftpd.
Configuring dnsmasq is pretty simple and straight forward. We have to modify two files /etc/hosts and /etc/dnsmasq.conf. Here is my /etc/hosts file
127.0.0.1 mokka.drunkenmonk.org mokka localhost ::1 mokka.drunkenmonk.org mokka localhost 192.168.2.1 bridge.mokka.drunkenmonk.org
The first line says ’127.0.0.1 is the ip address for mokka.drunkenmonk.org and also for ‘mokka’ and ‘localhost’ hostnames. I other words, ‘mokka’, ‘localhost’ and ‘mokka.drunkenmonk.org’ will resolve to 127.0.0.1 ip address.
‘mokka.drunkenmonk.org’ is a cononical name which contains two parts, ‘mokka’ as hostname and ‘drunkenmonk.org’ as domainname. So, by putting ‘drunkenmonk.org’, I’m also setting domainname for my system.
I really don’t know the proper way to configure domainname in my sabayon (in other words gentoo). From googling, I came to know that putting ‘domain drunkenmonk.org’ into /etc/resolv.conf is the right way to set domainname in a linux system. But, /etc/resolv.conf is such dynamic nowadays, we can’t be sure who will modify it later and I never see a router providing domainname in DHCP response (Not in BSNL’s ADSL routers for sure). So I endup putting my domainname in /etc/hosts file.
The second line is same like the first one, but for ipv6.
The third line says ’192.168.2.1′ is the ip address for ‘bridge.mokka.drunkenmonk.org’. I’ll come to this later. Now, take a look at my /etc/dnsmasq.conf
domain-needed bogus-priv no-resolv server=8.8.8.8 server=8.8.4.4 local=/drunkenmonk.org/ domain=drunkenmonk.org dhcp-range=interface:br0,192.168.2.2,192.168.2.254,255.255.255.0,1d mx-host=drunkenmonk.org,mail.drunkenmonk.org,30 cname=drunkenmonk.org,bridge.mokka.drunkenmonk.org cname=www.drunkenmonk.org,bridge.mokka.drunkenmonk.org cname=mail.drunkenmonk.org,bridge.mokka.drunkenmonk.org
‘domain-needed’ instructs dnsmasq to never forward DNS queries which don’t have domain part. It means, If you query for ‘nslookup google’ dnsmasq will never forward it to upstream server (In my case, to google’s nameservers).
‘bogus-priv’ tells dnsmasq never forward reverse-lookup queries which have local subnet’s ip rage to upstream. Which means ‘nslookup 10.0.0.1′ will not be forwarded to upstream instead dnsmasq will try to resolv itself. If it doesn’t find hostname for 10.0.0.1 in /etc/hosts or its dhcp leases, then it will send back ‘no such domain’ response.
‘no-resolv’ says dnsmasq will not read /etc/resolv.conf to get upsteram nameservers. Normally dnsmasq will read /etc/resolv.conf file to get upstream nameservers.
‘server=8.8.8.8′ and ‘server=8.8.4.4′ instucts dnsmasq to use google’s 8.8.8.8 and 8.8.4.4 as primary and secondary DNS nameservers (or in other words upstream nameservers).
‘local=/drunkenmonk.org/’ says that DNS queries with hostnames like ‘mokka.drunkenmonk.org’ should not be forwarded to upstream intead dnsmasq should resolve it from /etc/hosts or from its dhcp leases.
‘domain=drunkenmonk.org’ tells dnsmasq to send ‘domain=drunkenmonk.org’ in DHCP response so that machines configured through DHCP will come under ‘drunkenmonk.org’ domain.
‘dhcp-range=interface:br0,192.168.2.2,192.168.2.254,255.255.255.0,1d’ instructs dhsmasq to allocate ip address between 192.168.2.2 and 192.168.2.254 with netmask 255.255.255.0 for 1 day only to the DHCP requests coming from br0 interface. Which means, machines connected through br0 interface and asking for DHCP response will get ip address between 192.168.2.2 and 192.168.2.254 for 1 day. dnsmasq will not send response to requests coming from interfaces other than br0.
‘mx-host=drunkenmonk.org,mail.drunkenmonk.org,30′ tells dnsmasq to send ‘mail.drunkenmonk.org’ as MX response for ‘drunkenmonk.org’ domain. Which means, when you do ‘nslookup -q=MX drunkenmonk.org’ it will give ‘mail.drunkenmonk.org’ as response.
‘cname’ lines are like aliases. means when you do ‘nslookup http://www.drunkenmonk.org’; it will resolve to ‘bridge.mokka.drunkenmonk.org’ which resolves to ’192.168.2.1′ according to /etc/hosts file. So dnsmasq will send ’192.168.2.1′ as response.
Thats all from configuration stuff. These changes will take effect once you restart dnsmasq service. But this does not means that all your DNS queries will go through your machine’s dnsmasq daemon, If you have nameserver entry in /etc/resolv.conf, your machine’s DNS queries will go to /etc/resolv.conf nameservers. So, there is no use of using dnsmasq. You have to make sure you are not creating entries in /etc/resolv.conf. If you are a home user, make sure you select ‘Automatic (DHCP) Address only’ in ‘Method’ drop-down list inside ‘ipv4 settings’ tab in NetworkManger.
You can check that you are getting response from dhsmasq with below commands,
$ dig ANY drunkenmonk.org ; <> DiG 9.9.1-P2 <> ANY drunkenmonk.org ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39480 ;; flags: qr aa rd ra ad; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;drunkenmonk.org. IN ANY ;; ANSWER SECTION: drunkenmonk.org. 0 IN CNAME bridge.mokka.drunkenmonk.org. bridge.mokka.drunkenmonk.org. 0 IN A 192.168.2.1 ;; Query time: 0 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Mon Aug 6 00:15:16 2012 ;; MSG SIZE rcvd: 99
The line contains ‘SERVER:’ tell us that our DNS query is served by ’127.0.0.1′, means our local machine. Here is the command to verify MX record,
$ dig MX drunkenmonk.org ; <> DiG 9.9.1-P2 <> MX drunkenmonk.org ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 14301 ;; flags: qr aa rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;drunkenmonk.org. IN MX ;; ANSWER SECTION: drunkenmonk.org. 0 IN MX 30 mail.drunkenmonk.org. ;; Query time: 0 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Mon Aug 6 00:16:38 2012 ;; MSG SIZE rcvd: 71
If you are not introduced to Linux’s bridge interface, you may wonder what the hell is ‘br0′. Well , it is a virtual device in Linux which act like a physical network device (like eth0). I have particularly configured my dnsmasq to serve DHCP requests to whoever connect through my bridge interface. Howto use this ‘br0′ effectively to configure networking in my Qemu virtual machines is going to be my next article. Have a nice day!!
GNU Gettext – Yet Another Tutorial
Well, developing a simple C program is easy, but developing it in a internationalized way (Yeah!! all those l10n, m17n and i18n thingy) is not so easy unless you understand Autotools. However, understanding it may take some time (atleast it took some time for me). In this post, I’m trying to explain how I learnt it. It may be wrong way, but atleast I can recollect what I did today in future.
Lets just create a simple C project. Obviously without any doubt, it should be called ‘helloworld’. Lets just create the directory tree first.
$ mkdir -p helloworld/{src,man}
Switch to ‘helloworld/src’ and create two files ‘helloworld.h’ and ‘helloworld.c’
/* helloworld/src/helloworld.h */ #ifndef __HELLOWORLD__ #define __HELLOWORLD__ #define _GNU_SOURCE #include <stdio.h> #include <stdlib.h> #include <libintl.h> #include <locale.h> #define _(STRING) gettext(STRING) #endif
We have to include libintl.h to get ‘bindtextdomain(3)‘, ‘textdomain(3)‘ and ‘gettext(3)‘ functions. We have to include locale.h to get ‘setlocale(3)‘ function, let see why we need these functions,
setlocale()
Every glibc executable starts with the default locale called ‘C’. We use ‘setlocale()’ function to switch to different locale, this function takes two parameters ‘category’ and ‘locale’, ‘category’ indicates which locale variable we want to change and ‘locale’ contains what is the new value, If ‘locale’ is “”, setlocale() will get the value from the corresponding environment variable (see man page for more details).
bindtextdomain()
The gettext framework works as follows,
- Get all the english output strings from the sources and generate a .pot file
- Translate the english strings in the .pot file in different language and create .po file for each language
- Generate .gmo binary files from the .po file for each language
- Make the executable read corresponding translation from the .gmo file according to the locale settings each time it wants to print a message
To do the last step, we have to specify where the .gmo files are available. For that purpose, we use ‘bindtextdomain()’, it takes two arguments, ‘domainname’ and ‘dirname’. ‘domainname’ is the name we choose to group all our .gmo files under one place. Most of the time, we use the name of our project as ‘domainname’. ‘dirname’ is the common directory where different project’s .gmo files were placed. Usually it is ‘/usr/share/locale’.
textdomain()
we have to set the ‘textdomain’ so that executable will get the translated messages from the .gmo files correctly. ‘textdomain()’ takes only one argument ‘domainname’ which is the name of our project.
gettext()
Finally we have to wrap every output string to make them pass through ‘gettext()’ so that it can catch the correct translated string from .gmo files. We defined a macro ‘_()’ alias to ‘gettext()’ because we are lazy(aren’t we!?) to type ‘gettext()’ everytime.
So, here is the helloworld.c
/* helloworld/src/helloworld.c */
#include <helloworld.h>
int main(int argc, char *argv[])
{
setlocale(LC_ALL, "");
#ifdef ENABLE_NLS
bindtextdomain(PACKAGE, LOCALEDIR);
textdomain(PACKAGE);
#endif
printf(_("hello world\n"));
return(0);
}
Now, we have to replace ‘PACKAGE’ and ‘LOCALEDIR’ macros to the real values. Here comes autotools, automake can give real value to ‘PACKAGE’ at compile time and automake also have a way to define LOCALEDIR at compile time, Lets do autotools by creating following files,
# helloworld/src/Makefile.am bin_PROGRAMS = helloworld helloworld_SOURCES = helloworld.c helloworld.h DEFS += -DLOCALEDIR=\"$(localedir)\"
# helloworld/man/helloworld.1 helloworld :) !!! check after sometime to see the real man page
# helloworld/man/Makefile.am dist_man_MANS = helloworld.1
# helloworld/Makefile.am SUBDIRS = src man
We need to run ‘autoscan’ to generate ‘configure.scan’ file. Rename ‘configure.scan’ to ‘configure.ac’ and edit that file according to the project’s need. I can’t explain all the autoconf macros within this blog post, see the end of this blog post to get the links for further reading.
$ cd helloworld $ autoscan
Here is the customized ‘configure.ac’ file,
# helloworld/configure.ac
# -*- Autoconf -*-
# Process this file with autoconf to produce a configure script.
AC_INIT([helloworld], [0.1], [mokka at comedysite dot com])
AC_CONFIG_SRCDIR([src/helloworld.c])
# Automake init
AM_INIT_AUTOMAKE([foreign -Wall])
# Checks for programs.
AC_PROG_CC
AM_PROG_CC_C_O
# Gettext init
AM_GNU_GETTEXT_VERSION([0.18])
AM_GNU_GETTEXT([external])
# Checks for libraries.
# Checks for header files.
AC_CHECK_HEADERS([libintl.h locale.h stdlib.h])
# Checks for typedefs, structures, and compiler characteristics.
# Checks for library functions.
AC_CHECK_FUNCS([setlocale])
AC_CONFIG_FILES([Makefile
man/Makefile
src/Makefile])
AC_OUTPUT
Now we have to run ‘gettextize’ under ‘helloworld’ directory to put gettext settings into ‘configure.ac’ and ‘Makefile.am’.
$ cd helloworld $ gettextize
If things go well, you can see ‘helloworld/po’ directory and modifications into ‘configure.ac’ and ‘Makefile.am’. Now we can run ‘autoreconf’ to finish autotools procedure.
$ cd helloworld $ autoreconf --force --install --verbose
Now, switch to ‘helloworld/po’ and rename ‘Makevars.template’ to ‘Makevars’. Inside ‘Makevars’ file, you may have to give inputs to some variables, may be atleast to ‘MSGID_BUGS_ADDRESS’, Here is a way to add your email address to that variable
$ cd helloworld/po $ mv Makevars.template Makevars $ sed -i '/^MSGID/s/$/mokka at comedytime dot com/g' Makevars
Now, we need to add the source filenames to ‘POTFILES.in’, Here a way,
$ cd helloworld/po $ find ../src -name '*.c' -o -name '*.h' | sed 's/\.\.\///g' >> POTFILES.in
Time to compile,
$ cd helloworld $ ./configure $ make
You can see PACKAGE, LOCALEDIR macro definitions when make compile helloworld.c. As a programmer, your job is almost done.
Now switch yourself as a translator. go to ‘helloworld/po’ directory and generate a po file for your language using ‘msginit’, you have to provide ‘locale’ using -l option. You should know the languagecode and countrycode to construct ‘locale’ string. ‘msginit’ will ask for your email-id to put yourself into the translators list. Here I’m translating for ‘Tamil’ (ta_IN.utf8).
$ cd helloworld/po $ msginit -i helloworld.pot -o ta.po -l ta_IN.utf8
I edited ta.po file with gedit+ibus, translated the word “hello world\n” to “வனக்கம்\n’. Now, I have to add my language to LINGUAS file. LINGUAS file contains languagecodes which have corresponding translated .po file inside ‘helloworld/po’ directory.
# helloworld/po/LINGUAS ta
Now its time to generate binary .gmo file. Before that, We have to re-run ‘autoreconf’ to regenerate the helloworld/po/Makefile.in, because we updated LINGUAS file.
$ cd helloworld $ make distclean $ autoreconf --force --install --verbose $ cd po $ make update-gmo rm -f ta.gmo && /usr/bin/gmsgfmt -c --statistics --verbose -o ta.gmo ta.po ta.po: 1 translated message. $
If your translation don’t have any errors, you will see ’1 translated message’. Few more steps to achieve our goal, that is, creating distribution tarball and install our program to see the result.
$ cd helloworld $ make distclean $ make dist-bzip2 $ mkdir -p /tmp/buildir $ mv helloworld-0.1.tar.bz2 /tmp/builddir $ cd /tmp/builddir $ tar xvjf helloworld-0.1.tar.bz2 $ cd helloworld-0.1 $ ./configure --prefix="/tmp/destdir" $ make install $ LANG="ta_IN.utf8" /tmp/destdir/bin/helloworld வனக்கம் $ /tmp/destdir/bin/helloworld hello world $
Thats it. My ‘helloworld’ program can say “வனக்கம்” now. You can also make it to speak your favourite language!!
References
There is another beautiful tutorial for gettext available at oriya.sarovar.org.
pcapstreamer – A packet dumper
Hi guys,
Happy Holidays!!
This year is about to finish, thinking about this year, lot of things happened, love, break-up, home, health, mother, father, work and more importently passion. Well, all is well and life is moving ahead. Still I’m travelling alone, in my own path. (What the hell am I, this is suppose to be technical post, shit!! crap philosophy!!)
I got some free time and spent that time learning libpcap. For those who don’t know, it is used in most of the network monitoring/capturing tools in *nix world. Very powerful.
The ‘tcpdump(1)‘ command is one such tool which uses libpcap (actually they are the one who created libpcap from tcpdump) to dump information about packets. It has a robest filtering mechanism to narrow down packet capturing to specific packets.
While trying to understand filter expressions in tcpdump, I got an Idea, I thought why not just convert the bytes in packets to strings and print them in stdout, this way, we can see the exact bytes, so further processing can be done my other unix tools (like awk, perl etc.,).
So, I just wrote a tool called ‘pcapstreamer‘ to capture packets from linux’s ‘any’ psudo-interface. Its very simple tool, you need to run this tool as root user. It just dump packets, thats all. Here is an example, this shows one packet dumped into stdout.
$ sudo ./pcapstreamer [cl:76 l:76 t:20111226085033.641612] 00000000 00000000 00000011 00000100 00000000 00000110 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00001000 00000000 01000101 00000000 00000000 00111100 01000111 10001100 01000000 00000000 01000000 00000110 11110101 00101101 01111111 00000000 00000000 00000001 01111111 00000000 00000000 00000001 11100101 01100011 00010101 10110011 01111000 00011101 00100110 01010100 00000000 00000000 00000000 00000000 10100000 00000010 10000000 00011000 11111110 00110000 00000000 00000000 00000010 00000100 01000000 00001100 00000100 00000010 00001000 00001010 00000000 10100010 01100001 11011000 00000000 00000000 00000000 00000000 00000001 00000011 00000011 00000101
Here ‘cl:76′ and ‘l:76′ indicates ‘captured length’, ‘t:20111226085033.641612′ indicates ‘timestamp’ in localtime. Other strings are just pure raw packet.
Linux Cooked Header
To understand first 16 bytes, we need to understand ‘Linux Cooked Header‘. First 2 bytes “00000000 00000000′ or “0×00″ represents that this is an incoming packet. To understand the next 2 bytes, we need to refer linux’s ARPHRD_. 3rd and 4th bytes “00000011 00000100″ or “decimal 772″ indicates that this packet is coming into loopback interface. 5th and 6th bytes “00000000 00000110″ or “0×0006″ indicates the length of link-level address, the next 8 bytes (7th byte to 14th byte) represents the link-level address, however we should take only the next 6 bytes as link-level address, two more bytes (13th and 14th) are padded with zero. 15th and 16th bytes “00000000 00001000″ or “0×0008″ represents ‘ethertype‘ as ‘ip’, this tells us that this is an ‘ip’ packet. This ends the link-level header (data-link layer in OSI). We are now moving to ‘ip’ header (network layer in OSI)
IP Header
To understand details from 17th byte to 36th byte, we need to refer IP Header. Higher order 4 bits in 17th byte “0100″ or “0×4″ indicates that this ip packet is an ipv4 packet. Lower order 4 bits in 17th byte “0101″ or “0×5″ indicates IHL (Internet Header Length) usually this defaults to 5. 18th bytes represents ‘differentiated services’ usually 0. 19th and 20th bytes “00000000 00111100″ or “0x003c” or “Decimal 60″ represents remaining bytes count (CaptureLength minus Linux-Cooked-Header length). 21st and 22nd bytes “01000111 10001100″ indicates identification. Higher order 3 bits in 23rd and 24th bytes “010″ indicates that this packet is not fragmented, remaining 13 bits indicates fragment offset. 25th byte “01000000″ or “0×40″ or “Decimal 64″ indicates TTL value. 26th byte “00000110″ or “0×06″ indicates that this is a ‘tcp’ packet. 27th and 28th packets indicates Header Checksum. 29th to 32nd bytes indicates source ip address (127.0.0.1) and 33rd to 36th byte indicates destination ip address (127.0.0.1). This ends the ‘ip header’, we are now moving to ‘tcp’ header (Transport layer in OSI).
TCP Header
To understand details from 37th byte to 76th byte, we need to refer ‘TCP Header‘. 37th and 38th bytes “11100101 01100011″ or “decimal 58723″ indicates the source port number. 39th and 40th bytes “00010101 10110011″ or “decimal 5555″ indicates destination port number (means incoming packet is trying to connect port 5555). 41st to 44th byte indicates sequence number and 45th to 48th byte indicates sequence acknowledgement number. Higher order 4 bits in 49th and 50th byte “1010″ or “Decimal 10″ indicates Data offset, means there are 10*4=40 bytes in TCP header. Next 3 higher order bits are reserved in 49th byte 50th byte. Next 3 bits indicates ECN. Next 6 bits “000010″ or “0×02″ indicates that ‘SYN’ flag was set in Control bits. 51st and 52 bytes indicates window size, means the sender is willing to accept “10000000 00011000″ or “decimal 32792″ bytes in the response packet. 53rd and 54th bytes indicates checksum. 55th and 56th bytes indicates Urgent pointer, usually 0.
Options
Inside TCP header, bytes 57 to 76 contains value based on 50th byte(Data Offset). In this particular packet, 50th byte has (0xa), which means, TCP header in this packet contains totally 40bytes. Mandatory TCP fields (from 37th byte to 56th byte) are already discussed, but we have 20 more bytes to decode, these bytes are represented as ‘Options’ in TCP header. They may occur or they may not occur in a TCP packet. Mostly they occur in SYN packet.
Here, 57th byte (0×02) represents option-kind, 58th byte represents option-length (0×04). Both 57th and 58th bytes represents that 59th and 60th bytes “01000000 00001100″ or “0x400c” or “Decimal 16396″ indicates “Maximum Segment Size“. 61st byte (0×04) represents option-kind, 62nd byte (0×02) represents option-length, both bytes represents “SACK permitted“. 63rd byte (0×08) represents option-kind, 64th byte (0x0a) represents option-length, both bytes indicates that from 65th byte to 68th byte contains ‘TSVal‘ and from 69th byte to 72nd byte contains ‘TSecr‘. 73rd byte “0×01″ indicates option-kind as ‘No-Operation‘, 74th byte (0×03) indicates option-kind, 75 byte indicates (0×03) option-length, both bytes indicates that 76th byte (0×05) contains ‘WSOpt (Window Scale Option)‘, which means, the host which sent this packet can accept upto “32792 * (2^5)” or (windowsize[byte51&52] * (2^wsopt[byte76])) before sending ACK.
pcapstreamer with awk
To display only ICMP packets, we can use the following commandline
$ sudo ./pcapstreamer 2>/dev/null | awk '{if($26 ~ "00000001"){print $0;}}'
To display only SYN packets, we can use the following commandline
$ sudo ./pcapstreamer 2>/dev/null | awk '{ctrlbytes=$49$50; if(ctrlbytes ~ "^.......000010...$"){print $0;}}'
I hope this utility may be useful for newbies like me to learn networking. Have a great new year.
