Abusing the C preprocessor

August 29, 2011

Both tricks shown here are related with a change (by Peter Zijlstra) in the kmap_atomic() and kunmap_atomic() macros/functions. LWN has an excellent article about what that change involved. It basically ‘dropped’ support for atomic kmap slots, switching to a more general stack-based approach.

Now with this change, the number of arguments passed to the kmap_atomic() function changed too, and thus you end up with a huge patch covering all the tree, which fixed the issue (changing kmap_atomic(p, KM_TYPE) to kmap_atomic(p)).

But there was another way to go. Some C preprocessor magic.

#define kmap_atomic(page, args...) __kmap_atomic(page)

Yes, the C preprocessor supports va_args. :)
(which I found out when going through the reptyr code, but I’ll talk about it in an other post.)

Today, I saw a thread at the lkml, which actually did the cleanup I described. Andrew Morton responded:

I’m OK with cleaning all these up, but I suggest that we leave the back-compatibility macros in place for a while, to make sure that various stragglers get converted. Extra marks will be awarded for working out how to make unconverted code generate a compile warning

And Nick Bowler responded with a very clever way to do this (which involved abusing heavily the C preprocessor :P):

  #include <stdio.h>

  int foo(int x)
  {
     return x;
  }

  /* Deprecated; call foo instead. */
  static inline int __attribute__((deprecated)) foo_unconverted(int x, int unused)
  {
     return foo(x);
  }

  #define PASTE(a, b) a ## b
  #define PASTE2(a, b) PASTE(a, b)
  
  #define NARG_(_9, _8, _7, _6, _5, _4, _3, _2, _1, n, ...) n
  #define NARG(...) NARG_(__VA_ARGS__, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0)

  #define foo1(...) foo(__VA_ARGS__)
  #define foo2(...) foo_unconverted(__VA_ARGS__)
  #define foo(...) PASTE2(foo, NARG(__VA_ARGS__)(__VA_ARGS__))

  int main(void)
  {
    printf("%d\n", foo(42));
    printf("%d\n", foo(54, 42));
    return 0;
  }

The actual warning is printed due to the deprecated attribute of the foo_unconverted() function.

The fun part, however, is how we get to use the foo ‘identifier’/name to call either foo() or foo_uncoverted() depending on the number of arguments given. :)

The trick is to use the __VA_ARGS__ to ‘shift’ the numbers 9-0 in the NARG macro, so that when calling the NARG_ macro, _9 will match with the first __VA_ARGS__ argument, _8 with the second etc, and so n will match with actual number of arguments (I’m not sure I described it very well, but if you try doing it by hand, you’ll understand how it’s working).

Now that we have the number of arguments given to foo, we use the PASTE macro to ‘concatenate’ the number of the arguments with the function name, and the actual arguments given, and call the appropriate wrapper macro (foo1, foo2 etc).

Another interesting thing, which I didn’t know, is about argument expansion in macros. For macros that concatenate (##) or stringify (#) the arguments are not expanded beforehand. That’s why we have to use PASTE2 as a wrapper, to get the NARG() argument/macro fully expanded before concatenating.

Ok, C code can get at times a bit obfuscated, and yes you don’t have type safety etc etc, but, man, you can be really creative with the C language (and the C preprocessor)!
And the Linux kernel development(/-ers) prove just that. :)

A couple of months ago I found out about ketchup (credits to Daniel Drake, and his blog).

ketchup is an awesome utility/script, written by Matt Mackall in Python, which makes it very easy to manage kernel sources. You can very easily upgrade to a newer kernel version, downgrade to older releases, and even switch between different patchsets. The ketchup ebuild I found in Portage (and in every Linux distro I know about) was fetching the original and out-of-date version of ketchup. Steven Rostedt had pulled the original ketchup code (v0.9) into his git repo @ kernel.org. However, there were no commits/updates to ketchup for 1-2 years, I think.

So, I decided to cleanup some of the old trees that ketchup supported, but were no longer maintained, and add support for new trees (or some updated ‘versions’ of the old trees). I sent the patches to Steven Rostedt, and he proposed that I take over and maintain ketchup. :)

I cloned the ketchup git repo to Github, applied the patches I’d written, plus quite a lot of patches that the Debian ketchup package provided.

Now, with the Linux-3.0 release approaching, I tried to add (at least) partial support for the new 2 digit version numbers, but there are still some issues, which will hopefully get resolved once Linux-3.0 gets released, and the new versioning scheme gets standarized (for example the EXTRAVERSION Makefile variable will probably not get removed from 3.0, as it breaks some userspace utils, like uptime etc from procps utils, some depmod issues etc).

The new code for 3.x kernels is currently in the linux-3 branch, from which I took a snapshot and pushed it to Portage as dev-util/ketchup-1.1_beta. I will hopefully merge it back with master, after the first -stable release comes out (Linux-3.0.1), just to make sure that everything works.

Feel free to give it a try, and report any bugs/issues.

Hey!

July 9, 2011

I finally became a Gentoo Developer. :)

I’ll be helping the Gentoo Kernel Project, with bug fixing at first, and help with the maintenance of some of the kernel sources in the tree.

Many thanks to mpagano for mentoring me, tampakrap for his help with the quizzes, and of course hwoarang, who had no problem to do all of the review sessions during his vacations. :)

I had blogged about this some time ago. The configuration I described in that post worked fine on my laptop, with Debian installed, but when I tried it on my Desktop, where I use Gentoo, it wouldn’t work.

It took me *3 days* of ‘debugging’, until I was able to find why that happened!

I tried various changes to the iptables and iproute2 configuration, giving more hints to both utilities in order to use the correct routing table, mark the packets correctly etc, but it still wouldn’t work.

After a lot of time tweaking the configuration, without results, I saw that, although ping -Ieth0 ${VPN_SERVER}, didn’t ‘work’ (with openvpn running, and tap0 configured with the correct address/netmask), I could see with tcpdump the ‘ECHO REPLY’ packets sent by the VPN server, with correct source and destination addresses.

After stracing the ping command, I saw that when ping issued a recvmsg syscall, recvmsg returned with -EAGAIN. So, now I know that the packets do arrive to the interface with correct addresses, but they couldn’t ‘reach’ the upper network stacks of the kernel.

The problem was that both machines were running vanilla kernels, so I couldn’t blame any Debian or Gentoo specific patches. But since I knew that the problem was in the kernel, I tried to see if any kernel .config options, regarding NETFILTER, and multiple routing tables didn’t match between the two configs. But I couldn’t find anything that could cause that ‘bug’.

So since the kernel sources are the same, and I can’t find anything in the .configs that could cause the problem, I try tweaking some /proc/sys/net ‘files’, although I couldn’t see why these would differ between the two machines. And then I saw some /proc/sys/net/ipv4/ files in Gentoo, that didn’t show up in Debian (/proc/sys/net/ipv4/cipso*).

I googled to find what cipso is, and I finally found out that it was part of the NetLabel project. CIPSO (Common IP Security Option) is an IETF draft (it’s quite old actually) and is implemented like a ‘security module’ in the Linux Kernel, and it was what it caused the problem, probably because it tried to do some verification on the inbound packets, which failed, and therefore the packets were ‘silently’ dropped. LWN has an article with more infromation about packet labeling and CIPSO, and there’s also related Documentation in the Linux Kernel.

make defconfig enbales Netlabel, but Debian’s default configuration had it disabled, and that’s why Openvpn/iproute2/iptables configuration worked with Debian, but failed on Gentoo.

Instead of compiling a new kernel, one can just do

echo 0 > /proc/sys/net/ipv4/cipso_rbm_strict_valid

and disable CIPSO verification on inbound packets, so that multiple routing tables and packet marking work as expected.

A couple of days ago, we did some presentations about DNS at a FOSS NTUA meeting.

I prepared a presentation about DNS tunneling and how to bypass Captive Portals at Wifi Hotspots, which require authentication.
(We want to do another presentation, to test ICMP/ping tunnel too ;)).

I had blogged on that topic some time ago.
It was about time for a test-drive. :P

I set up iodine, a DNS tunneling server(and client), and I was ready to test it, since I would be travelling with Minoan Lines the next day.

I first did some tests from my home 24Mbps ADSL connection, and the results weren’t very encouraging. Although the tunnel did work, and I could route all of my traffic through the DNS tunnel, and over a second OpenVPN secure tunnel, bandwidth dropped to ~30Kbps, when using the NTUA FTP Server, through the DNS tunnel.
(The tunnel also worked with the NTUA Wifi Captive Portal, although at first we had some ‘technical issues’, ie I hadn’t set up NAT on the server to masquarade and forward the traffic coming from the tunnel :P).

The problem is that the bandwidth of the Minoan Lines(actually Forthnet ‘runs’ it afaik) Wifi(not inside the ‘local’ network of course) was ~30Kbps(terrible, I know), without using DNS tunneling. So, I wasn’t very optimistic. (I think they have some satelite connection, or something like that from the Wifi to the Internet).

When I was on the ship, I tried to test it. At first, I encountered another technical issue(the local DNS had an IP inside the Wifi local network, and due to NAT the IP our server was ‘seeing’, was different than the IP of the DNS packets, so we had to run iodined with the -c flag). Luckily, FOSS NTUA members(who had root access on the computer running iodined) are 1337 and fixed that in no time. :P

And at last, I had a ‘working’ DNS tunnel, but with extremely high ping times(2sec RTT) to the other end of the tunnel, and when I tried to route all traffic through the tunnel I had a ridiculous 22sec RTT to ntua.gr. Of course even browsing the Web was impossible, since all the HTTP requests timed out before an answer could reach my laptop. :P

However, because I am a Forthnet customer(for my ADSL connection), I was able to use my Username/Password of my home ADSL connection, and have free access to the Internet, from their hotspot(with the amaing bandwidth of ~30Kbps :P). At least they do the authentication over SSL. :P

Although DNS tunneling didn’t really work in this case(the tunnel itself worked, but due to the bandwidth being so low, I didn’t have a ‘usable’ connection to the Internet), I think that in other hotspots, which provide better bandwidth/connection, it can be a very effective way to bypass the authentication and use them for free. ;)

Probably, there’ll be a Part 3, with results from bandwidth benchmarks inside the NTUA Wifi, and maybe some ICMP tunneling stuff.

Cheers! :)

my first kernel patch!

March 11, 2011

Here it is! :P

Well, the patch itself isn’t a big deal, since I didn’t write any code. It was a cleanup of asm-offsets.

Afaik, Linux has quite a lot of assembly code, which needs the offsets of various struct members. Of course, assembly code(or even toplevel inline assembly) cannot use the offsetof marco. That’s also the case for some C constants, eg:

#define PAGE_SIZE (1UL << PAGE_SHIFT)) 

Thus, these offsets and constants are ‘calcualted’ at build time as ‘absolute values’ so that gas will be ok. :P
Using the macros defined in incldue/linux/kbuild.h

#define DEFINE(sym, val) \
        asm volatile("\n->" #sym " %0 " #val : : "i" (val))

#define BLANK() asm volatile("\n->" : : )

#define OFFSET(sym, str, mem) \
	DEFINE(sym, offsetof(struct str, mem))

#define COMMENT(x) \
	asm volatile("\n->#" x)

the asm-offsets.c is used to create a ‘special’ asm fle, which is then parsed by the kernel build system, in order to create the include/generated/asm-offsets.h file.

However, a patch introduced two new macros in include/linux/const.h

#ifdef __ASSEMBLY__
#define _AC(X,Y)	X
#define _AT(T,X)	X
#else
#define __AC(X,Y)	(X##Y)
#define _AC(X,Y)	__AC(X,Y)
#define _AT(T,X)	((T)(X))
#endif

so that some constants defined in C work with gas too.

And now, the PAGE_SIZE is defined as

#define PAGE_SIZE       (_AC(1,UL) << PAGE_SHIFT

and thus, the PAGE_SIZE_asm defined in x86/asm-offsets.c, and used in some places in x86 code was no longer needed(that’s also the case with PAGE_SHIFT and THREAD_SIZE).

So, I deleted PAGE_SIZE_asm/PAGE_SHIFT_asm/THREAD_SIZE_asm from x86/kernel/asm-offsets.c, and replaced them with their non-asm counterparts, in the code that used them.

I posted it to lkml(after some hours experimenting with git format-patch and git send-email :P), and it got accepted. :)

RFC mania

January 22, 2010

I had to do an SNMP-related excercise for the Network Management Lab. We had to write a MIB(Message Information Base) for a firewall, to describe the filters and the rules of the firewall.
The MIB should be written in SNMPv2 SMI, so I read some RFCs.
I never liked the RFCs, and now I think that they’re even more disgusting. :P
Actually, I think that people who are involved with the whole process of the RFCs have serious personal problems(just kidding :P).
And to prove that I have a point, a friend of mine reminded me of an epic RFC.
RFC 1149, or IP over Avian Carriers!!!!
And that’s not the worst part. There’s more!
Some people did an actual implementation of the RFC!
I knew about the RFC but not that there was an “implementation”. :P
About 10 years ago.
Link 1 and Link 2.
The highlight(besides the pigeons ofcourse) was the source code, and the ping times.

Ok, in fact, I would say that the whole thing was fun and maybe interesting, but the RFCs are still disgusting. :P
Except for RFCs like these :P
:D

sshd + reverse DNS lookup

October 19, 2009

This post is mainly for ‘self reference’, in case something like this happens again.

According to the sshd man page, by default, sshd will perform a reverse DNS lookup, based on the client’s IP, for various reasons.

A reverse DNS lookup is used in order to add the hostname to the utmp file, that keeps track of the logins/logouts to the system. One way to ‘switch it off’ is by using the -u0 option when stating sshd. The -u option is used to specify the size of the field of the utmp structure that holds the remote host name.

A reverse lookup is also performed when the configuration(or the authentication mechainsm used) requires such a lookup. The HostBasedAuthentication auth mech, a “from=hostname” option in the .authorized_keys file, or the AllowUsers/DenyUsers option that includes hostnames, in the sshd_config, require a reverse DNS lookup.

Btw, the UseDNS option in the sshd_config, which I think is enable by default, will not prevent sshd from doing a reverse lookup, for the above mentioned reasons. However, if this option is set to ‘No’, sshd will not try to verify that the resolved hostname maps back to the same IP that the client provided(adding an extra ‘layer’ of security).

So, the point is that if for some reason the ‘primary’ namserver in the resolv.conf is not responding, you’ll experience a lag when trying to login using ssh, which can be confusing if you don’t know the whole reverse DNS story.

Another thing that I hadn’t thought before I learned about sshd reverse lookups, is that a DNS problem can easily ‘lock you out’ of a computer, if you use hostname based patterns with TCP wrappers(hosts.allow, hosts.deny). And maybe this can explain some “Connection closed by remote host” errors, when trying to login to a remote computer. :P

rsnapshot tips&tricks

July 19, 2009

rsnapshot is a great application for taking backups. It uses rsync and hard links, and makes backup management very easy. It comes with a nice perl script, rsnapreport, which reads the output of the rsync commands used by rsnapshot, and prints a useful report, with the stats of each rsync command.

For detailed info about rsnapshot, you can visit the website of the project.

A typical configuration would set up a cronjob, in which the output of the rsnapshot sync command(or rsnapshot daily, if the sync_first option is not enabled), is piped to rsnapreport, and the output of rsnapreport is piped to a CLI SMTP client, to send us a mail with the stats of the sync operation.(sendEmail is a very nice SMTP client ;) ).

However, if you’re taking backups from multiple machines, the sync operation can last longer than expected. So, if the datablock timeout in our SMTP server isn’t large enough, we will never get an email.

This is solved if we use a wrapper script for the rsnapshot sync operation. We use that script for the cronjob, and inside the script we have something like this:

#!/bin/sh
rsnapshot sync > /tmp/rsync_stats 2>&1
cat /tmp/rsync_stats | rsnapreport.pl | sendEmail -f backup@foo -t user@bar -u rsnapshot report
rm -f /tmp/rsync_stats

And problem solved! ;)

Follow

Get every new post delivered to your Inbox.

Join 276 other followers