If you uninstall stuff with apt-get remove sometimes Debian does not removes config files and they also take up space.
One approach is to use apt-get purge.
An alternative is as follows:
This command lists all the packages which are removed but still have config files on your system:
dpkg --list | grep '^rc ' rc binutils 2.20.1-15 The GNU assembler, linker and binary utilities rc dbus 1.2.24-3 simple interprocess messaging system rc dpkg-dev 1.15.8.5 Debian package development tools rc erlang-base 1:14.a-dfsg-2 Erlang/OTP virtual machine and base applications rc fakeroot 1.14.4-1 Gives a fake root environment
Now, just to make sure check the output and then remove the config files with this command:
dpkg --list | grep '^rcb' | awk '{ print $2 }' | xargs dpkg -P