Differences

This shows you the differences between two versions of the page.

--- curl:perform_imap_queries_using_curl [2019/11/27 00:07] – removed peter
+++ curl:perform_imap_queries_using_curl [2020/09/25 13:02] (current) – 192.168.1.1
@@ Line 1: / Line 1: @@
+====== Curl - Perform imap queries using Curl ======
+If you have an IMAP or IMAPS server you can "read your email" using Curl.  To get started you'll first of all need to know:
+  * Your mail-server address.
+  * Your mail login.
+  * Your mail password.
+----
+===== Listing Folders =====
+The initial example will connect us to the IMAP server at imap.example.com, using username bobby and password tables:
+<code bash>
+curl --url "imap://mail.example.com/" --user "bobby:tables"
+</code>
+<WRAP info>
+If you leave out the password from the **<nowiki>--user</nowiki>** argument, curl will prompt you for the password before it preforms the request. This way you're not leaking your password to your shell history or the process list.
+Curl also supports .netrc lookup of user:password with -n, if the **<nowiki>--user</nowiki>** argument is left off.
+</WRAP>
+returns
+<code bash>
+..
+..
+* LIST (\HasNoChildren) "/" xen-users
+* LIST (\HasNoChildren) "/" mentors-debian-org
+* LIST (\HasNoChildren) "/" fairshare
+* LIST (\HasNoChildren) "/" INBOX
+</code>
+Here we see that we've connected, and received a list of folders, including "xen-users", "fairshare", and the "INBOX".
+If your mail-server is running over SSL then instead of using **<nowiki>imap://</nowiki>** you should set the schema to **<nowiki>imaps://</nowiki>**, it may be that you're using a self-signed certificate in that case you'd add **<nowiki>--insecure</nowiki>** to avoid checking the certificate trust-chain.
+This would look like so:
+<code bash>
+curl --insecure  --url "imaps://mail.example.com/" --user "bobby:tables"
+</code>
+----
+===== Discovering Messages =====
+With the previous example we looked at listing mailboxes.  What if we wanted to actually view a message?
+To fetch a message we need the identifier of the message to fetch - so we need to find out how many messages exist, as message-IDs are sequential.
+To see how many messages exist in the folder "People-Steve" we'd run this:
+<code bash>
+curl --insecure \
+    --url "imaps://mail.example.com/" \
+    --user "bobby:tables" \
+    --request "EXAMINE People-Steve"
+</code>
+returns:
+<code bash>
+* OK [PERMANENTFLAGS ()] Read-only mailbox.
+* 9465 EXISTS
+* 3266 RECENT
+* OK [UIDVALIDITY 1373146046] UIDs valid
+* OK [UIDNEXT 9468] Predicted next UID
+</code>
+This tells us that there are 9465 messages.  So we can probably assume that fetching a message with each of these IDs will work: 1, 2, 3, ... 9465, & 9465.
+----
+===== Fetching A Single Message =====
+Fetching a message is simple if you know both the folder from which it comes and the ID of the message you wish to retrieve.
+We've already shown that the folder "People-Steve" contains nearly ten-thousand messages, so this next example will show us fetching the message with ID 512:
+<code bash>
+curl --insecure \
+    --url "imaps://mail.example.com/People-Steve;UID=512" \
+    --user "bobby:tables"
+</code>
+returns:
+<code bash>
+..
+X-HELO: mail.cs.helsinki.fi
+X-REMOTE-IP: 128.214.9.1
+X-REMOTE-HOST: courier.cs.helsinki.fi
+..
+..
+I just accidentally someone! (see facebook)
+..
+</code>
+Fetching the message includes both the headers and the body.  If you want to fetch just the headers, or a subset, things are a little fiddlier due to URL-encoding.  This example gets the headers "to", "from", "date", and "subject":
+<code bash>
+curl --insecure \
+    --url "imaps://mail.example.com/People-Steve;UID=512;SECTION=HEADER.FIELDS%20(DATE%20FROM%20TO%20SUBJECT)" \
+    --user "bobby:tables"
+</code>
+returns:
+<code bash>
+Date: Sat, 14 Apr 2012 14:13:41 +0300 (EEST)
+From: Steve Kemp <steve@steve.org.uk>
+To: Steve Kemp <steve@steve.org.uk>
+Subject: Re: Fancy a cake?
+</code>
+We could have avoided the use of URL-encoding and instead sent a custom-request, like so:
+<code bash>
+curl --insecure --verbose \
+    --url "imaps://imap.example.com/People-Steve" \
+    --user "bobby:tables" \
+    --request "fetch 512 BODY.PEEK[HEADER.FIELDS (Subject)]"
+</code>
+returns:
+<code bash>
+..
+Subject: Re: Fancy a cake?
+..
+</code>
+The reason for avoiding custom-requests where possible is that when you're using the curl API programatically you'll discover that responses are not decoded - which is a known issue.
+In the example above we'd have received zero output unless/until we added the **<nowiki>--verbose</nowiki>** flag.  Precisely because curl has received the output from the IMAP-server but not decoded it and presented it to us.
+----
+===== Simple Shell Scripts =====
+As a quick hack the following shell-script will dump the subject of the first ten messages in the given mailbox:
+<code bash>
+#!/bin/sh
+# Dump the subject of the first ten messages in the folder.
+for id in `seq 1 10` ; do
+    echo "Message ${id}"
+    curl --insecure \
+        --url "imaps://mail.example.com/People-Steve;UID=${id};SECTION=HEADER.FIELDS%20(SUBJECT)" \
+        --user "bobby:tables"
+done
+</code>
+This takes no account of the maximum message-ID.  You could just keep going indefinitely, to dump all subjects and stop on error:
+<code bash>
+#!/bin/sh
+# Dump the subject of all messages in the folder.
+id=1
+while true  ; do
+    echo "Message ${id}"
+    curl --insecure \
+        --url "imaps://mail.example.com/People-Steve;UID=${id};SECTION=HEADER.FIELDS%20(SUBJECT)" \
+        --user "bobby:tables" || exit
+    id=`expr $id + 1`
+done
+</code>
+----
+===== CurlMail Script =====
+<code bash>
+#!/bin/bash
+#                       /usr/local/bin/curlmail
+#  http://crystalfaeries.net/posix/bin/curlmail
+# celeste crystalfaery 2016-02-24 22:27:10+00:00
+# https://debian-administration.org/article/726/Performing_IMAP_queries_via_curl
+# read(only) IMAPS INBOX  (using ~/.netrc for password)
+# (machine example.org login user password my_password)
+# 1st argument is machine name (else talk to self if not specified)
+# 2..n argument(s) (optional) are message number(s) to view
+# e.g.: curlmail example.org 3 1 4
+help=10 # this line number of file -1
+# NOTE: remove the "--insecure" argument to curl if server(s) have real certificate vs self-signed certificate
+# NOTE: we do not sanitize the server argument, so as-is don't make this into a cgi
+function cleanup {
+  if [ -e /tmp/$$ ]; then
+    rm	/tmp/$$		# cleanup our temporary file
+  fi
+}
+trap cleanup INT TERM			# cleanup our temporary file
+case $# in
+)
+  exec	$0 `hostname`		# talk to myself
+  exit	-1			# exec doesn't return, right?
+  ;;
+)
+  case "$1" in
+    -v)
+	tail -n +4 $0 | head -n 1
+	;;
+    --version)
+	tail -n +4 $0 | head -n 1
+	;;
+    -h)
+	head -n $help $0 | tail -n +6
+	;;
+    --help)
+	head -n $help $0 | tail -n +6
+	;;
+    *)
+	let uid=0
+	while let uid=$uid+1
+	do	# iterate available messages
+	  echo -n "${uid}: "									>> /tmp/$$
+	  chmod 600										   /tmp/$$
+	  curl -# --insecure -n --url \
+		"imaps://${1}/INBOX;UID=${uid};SECTION=HEADER.FIELDS%20(DATE%20FROM%20TO%20SUBJECT)"	>> /tmp/$$ 2>/dev/null || break
+	done
+	sed 's/\n\n/\n/g'										<  /tmp/$$ | less
+	rm												   /tmp/$$
+	;;
+    esac
+      exit
+      ;;
+*)
+  server="$1"
+  shift
+  while [ $# != 0 ]
+  do  # iterate requested messages
+    message=`echo "$1" | tr -d -c '[:digit:]'`	# sanitize input
+    shift
+    rm									   /tmp/$$ 2>/dev/null
+    echo -n "${message}: "							>> /tmp/$$
+    chmod 600								   /tmp/$$
+    curl -# --insecure -n --url "imaps://${server}/INBOX;UID=${message}"	>> /tmp/$$ 2>/dev/null
+    sed 's/\n\n/\n/g'							<  /tmp/$$ | less
+    rm									   /tmp/$$
+  done
+  exit
+  ;;
+esac
+</code>
+----
+===== References =====
+http://crystalfaeries.net/fae/curlmail.html