Comprehensive IMAP Implementation Plan#

This document consolidates all RFC implementation plans from spec/ and lib/imap/PLAN.md into a single, prioritized implementation roadmap. Per the design goals, we favor OCaml variants over strings and do not require backwards compatibility.

Executive Summary#

The ocaml-imap library implements IMAP4rev2 (RFC 9051) with several extensions. This plan covers:

• P0: Critical fixes and core infrastructure
• P1: Core protocol compliance
• P2: Extension support (SORT/THREAD, QUOTA, etc.)
• P3: Advanced features (UTF-8, CONDSTORE/QRESYNC)
• P4: Polish (documentation, unified flag library)

Phase 0: Critical Fixes (P0)#

These are blocking issues that need immediate attention.

0.1 Fix SEARCH Response Parsing (Client Library)#

Source: lib/imap/PLAN.md - P0 Broken Functionality

Problem: search function always returns empty list - response is never parsed.

Files:

• lib/imap/read.ml - Add SEARCH response parsing
• lib/imap/client.ml:536-544 - Fix to read response

Implementation:

(* In read.ml - add case for SEARCH response *)
| "SEARCH" ->
    let rec parse_numbers acc =
      match R.peek_char r with
      | Some ' ' -> sp r; parse_numbers (number r :: acc)
      | Some c when c >= '0' && c <= '9' -> parse_numbers (number r :: acc)
      | _ -> List.rev acc
    in
    let nums = parse_numbers [] in
    crlf r;
    Response.Search nums

Tests (test/test_read.ml):

let test_search_response () =
  let resp = parse "* SEARCH 2 4 7 11\r\n" in
  Alcotest.(check (list int)) "search" [2; 4; 7; 11]
    (match resp with Response.Search nums -> nums | _ -> [])

let test_search_empty () =
  let resp = parse "* SEARCH\r\n" in
  Alcotest.(check (list int)) "empty search" []
    (match resp with Response.Search nums -> nums | _ -> [-1])

0.2 Parse BODY/BODYSTRUCTURE Responses#

Source: lib/imap/PLAN.md - P1 Incomplete Core Features

Problem: FETCH responses with BODY/BODYSTRUCTURE fall back to empty flags.

Files:

• lib/imap/read.ml:284-302 - Add BODY/BODYSTRUCTURE parsing
• lib/imap/body.ml - Body structure types (may need new file)

Implementation: Parse nested multipart MIME structures recursively.

Tests:

let test_body_structure () =
  let resp = parse {|* 1 FETCH (BODYSTRUCTURE ("TEXT" "PLAIN" ("CHARSET" "UTF-8") NIL NIL "7BIT" 1234 56))|} in
  (* verify body structure parsed correctly *)

0.3 Parse BODY[section] Literal Responses#

Source: lib/imap/PLAN.md - P1

Problem: Cannot read actual message content from FETCH.

Implementation: Parse section specifiers and literal data:

(* Patterns: BODY[HEADER], BODY[TEXT], BODY[1.2.MIME], BODY[section]<origin> *)

Phase 1: Core Protocol Compliance (P1)#

1.1 Complete ESEARCH Support (RFC 4731)#

Source: spec/PLAN-rfc4731.md

Current State: Response type exists, parsing not implemented.

Tasks:

1. Add ESEARCH response parsing to lib/imap/read.ml
2. Add search return options to Command type
3. Add serialization for RETURN (MIN MAX COUNT ALL)
4. Add client API functions

Types (use variants, no strings):

type search_return_opt =
  | Return_min
  | Return_max
  | Return_all
  | Return_count

type esearch_result =
  | Esearch_min of int
  | Esearch_max of int
  | Esearch_count of int
  | Esearch_all of Seq.t

Tests:

let test_esearch_parsing () =
  let resp = parse "* ESEARCH (TAG \"A282\") MIN 2 COUNT 3\r\n" in
  assert (resp = Response.Esearch {
    tag = Some "A282";
    uid = false;
    results = [Esearch_min 2; Esearch_count 3]
  })

1.2 Parse APPENDUID/COPYUID Response Codes#

Source: lib/imap/PLAN.md - P1

Files: lib/imap/read.ml:169-228

Implementation:

(* Add to response_code parsing *)
| "APPENDUID" ->
    sp r;
    let uidvalidity = number32 r in
    sp r;
    let uid = number32 r in
    Code.Appenduid (uidvalidity, uid)
| "COPYUID" ->
    sp r;
    let uidvalidity = number32 r in
    sp r;
    let source_uids = parse_uid_set r in
    sp r;
    let dest_uids = parse_uid_set r in
    Code.Copyuid (uidvalidity, source_uids, dest_uids)

1.3 UNSELECT Capability Advertisement#

Source: spec/PLAN-rfc3691.md

Status: Fully implemented except capability not advertised.

Fix (lib/imapd/server.ml):

let base_capabilities_pre_tls = [
  (* existing *)
  "UNSELECT";  (* RFC 3691 - already implemented *)
]

1.4 SPECIAL-USE Support (RFC 6154)#

Source: spec/PLAN-rfc6154.md

Current: Types exist, capability not advertised, flags not returned.

Tasks:

1. Add SPECIAL-USE to capabilities
2. Return special-use flags in LIST responses
3. Map standard mailbox names to attributes

Types (already exist, ensure completeness):

type special_use =
  | All | Archive | Drafts | Flagged | Important
  | Junk | Sent | Trash
  | Snoozed | Scheduled | Memos  (* draft-ietf-mailmaint *)

Tests:

let test_list_special_use () =
  (* LIST "" "*" should return \Drafts on Drafts mailbox *)

Phase 2: Extension Support (P2)#

2.1 SORT/THREAD Extension (RFC 5256)#

Source: spec/PLAN-rfc5256.md

Scope: Large feature - server-side sorting and threading.

2.1.1 Thread Module Types#

New file: lib/imap/thread.ml

type algorithm =
  | Orderedsubject  (** Group by subject, sort by date *)
  | References      (** Full JWZ threading algorithm *)
  | Extension of string

type 'a node =
  | Message of 'a * 'a node list
  | Dummy of 'a node list

type 'a t = 'a node list

2.1.2 Base Subject Extraction#

New file: lib/imap/subject.ml

Implements RFC 5256 Section 2.1 algorithm:

1. Decode RFC 2047 encoded-words
2. Remove Re:, Fw:, Fwd: prefixes
3. Remove [blob] prefixes
4. Remove (fwd) trailers
5. Unwrap [fwd: ...] wrappers

val base_subject : string -> string
val is_reply_or_forward : string -> bool

2.1.3 Sent Date Handling#

New file: lib/imap/date.ml

type t

val of_header : string -> t option
val of_internaldate : string -> t
val sent_date : date_header:string option -> internaldate:string -> t
val compare : t -> t -> int

2.1.4 Server-Side SORT Handler#

File: lib/imapd/server.ml

1. Implement sort key extraction
2. Implement comparison by criteria
3. Return SORT response

2.1.5 Threading Algorithms#

New file: lib/imapd/thread.ml

1. orderedsubject - simple subject-based grouping
2. references - full JWZ algorithm (6 steps)

Tests:

let test_base_subject () =
  assert (Subject.base_subject "Re: test" = "test");
  assert (Subject.base_subject "Re: Re: test" = "test");
  assert (Subject.base_subject "[PATCH] Re: [ocaml] test" = "test");
  assert (Subject.base_subject "[fwd: wrapped]" = "wrapped")

let test_orderedsubject () =
  (* Test grouping by subject *)

let test_references_threading () =
  (* Test parent/child relationships *)

2.2 QUOTA Extension (RFC 9208)#

Source: spec/PLAN-rfc9208.md

2.2.1 Protocol Types#

File: lib/imapd/protocol.ml

type quota_resource =
  | Quota_storage           (** KB of storage *)
  | Quota_message           (** Number of messages *)
  | Quota_mailbox           (** Number of mailboxes *)
  | Quota_annotation_storage

type quota_resource_info = {
  resource : quota_resource;
  usage : int64;
  limit : int64;
}

(* Commands *)
| Getquota of string
| Getquotaroot of mailbox_name
| Setquota of { root : string; limits : (quota_resource * int64) list }

(* Responses *)
| Quota_response of { root : string; resources : quota_resource_info list }
| Quotaroot_response of { mailbox : mailbox_name; roots : string list }

2.2.2 Storage Backend Interface#

File: lib/imapd/storage.mli

val get_quota_roots : t -> username:string -> mailbox_name -> string list
val get_quota : t -> username:string -> string -> (quota_resource_info list, error) result
val set_quota : t -> username:string -> string -> (quota_resource * int64) list -> (quota_resource_info list, error) result
val check_quota : t -> username:string -> mailbox_name -> additional_size:int64 -> bool

2.2.3 Server Handlers#

Implement handle_getquota, handle_getquotaroot, handle_setquota.

Add quota checks to APPEND/COPY/MOVE:

if not (Storage.check_quota ...) then
  send_response flow (No { code = Some Code_overquota; ... })

Tests:

let test_getquotaroot () =
  (* GETQUOTAROOT INBOX returns quota info *)

let test_quota_exceeded () =
  (* APPEND fails with OVERQUOTA when over limit *)

2.3 LIST-EXTENDED (RFC 5258)#

Source: spec/PLAN-rfc5258.md

Types:

type list_select_option =
  | List_select_subscribed
  | List_select_remote
  | List_select_recursivematch
  | List_select_special_use  (* RFC 6154 *)

type list_return_option =
  | List_return_subscribed
  | List_return_children
  | List_return_special_use

type list_extended_item =
  | Childinfo of string list

type list_command =
  | List_basic of { reference : string; pattern : string }
  | List_extended of {
      selection : list_select_option list;
      reference : string;
      patterns : string list;
      return_opts : list_return_option list;
    }

Tasks:

1. Update grammar for extended LIST syntax
2. Add \NonExistent and \Remote attributes
3. Implement subscription tracking in storage
4. Handle RECURSIVEMATCH with CHILDINFO
5. Add LIST-EXTENDED capability

Phase 3: Advanced Features (P3)#

3.1 UTF-8 Support (RFC 6855)#

Source: spec/PLAN-rfc6855.md

3.1.1 Session State Tracking#

type session_state = {
  utf8_enabled : bool;
  (* ... *)
}

3.1.2 UTF-8 Validation#

New file: lib/imapd/utf8.ml

val is_valid_utf8 : string -> bool
val has_non_ascii : string -> bool
val is_valid_utf8_mailbox_name : string -> bool

3.1.3 ENABLE Handler Update#

Track UTF8=ACCEPT state, reject SEARCH with CHARSET after enable.

3.1.4 UTF8 APPEND Extension#

Parse UTF8 (literal) syntax for 8-bit headers.

Tests:

let test_utf8_validation () =
  assert (Utf8.is_valid_utf8 "Hello");
  assert (Utf8.is_valid_utf8 "\xe4\xb8\xad\xe6\x96\x87");
  assert (not (Utf8.is_valid_utf8 "\xff\xfe"))

3.2 CONDSTORE/QRESYNC (RFC 7162)#

Source: lib/imap/PLAN.md - P2, PLAN.md - Phase 2.2

3.2.1 CONDSTORE Types#

(* Fetch items *)
| Modseq
| Item_modseq of int64

(* Response codes *)
| Highestmodseq of int64
| Nomodseq
| Modified of Seq.t

(* Command modifiers *)
type fetch_modifier = { changedsince : int64 option }
type store_modifier = { unchangedsince : int64 option }

3.2.2 Storage Backend#

Add modseq to message type and mailbox state:

type message = {
  (* existing *)
  modseq : int64;
}

type mailbox_state = {
  (* existing *)
  highestmodseq : int64;
}

3.2.3 QRESYNC#

type qresync_params = {
  uidvalidity : int32;
  modseq : int64;
  known_uids : Seq.t option;
  seq_match : (Seq.t * Seq.t) option;
}

(* Response *)
| Vanished of { earlier : bool; uids : Seq.t }

Phase 4: Polish and Infrastructure (P4)#

4.1 RFC 5530 Response Code Documentation#

Source: spec/PLAN-rfc5530.md

All 16 response codes already implemented. Add OCamldoc citations.

4.2 Unified Mail Flag Library#

Source: spec/PLAN-unified-mail-flag.md

Create shared mail-flag library for IMAP/JMAP:

mail-flag/
├── keyword.ml      # Message keywords (typed variants)
├── system_flag.ml  # IMAP \Seen, \Deleted, etc.
├── mailbox_attr.ml # Mailbox attributes/roles
├── flag_color.ml   # Apple Mail flag colors
├── imap_wire.ml    # IMAP serialization
└── jmap_wire.ml    # JMAP serialization

4.3 Infrastructure Improvements#

Source: PLAN.md - Phase 1

1. Replace Menhir with Eio.Buf_read - Pure functional parser
2. Integrate conpool - Connection pooling for client
3. Add bytesrw streaming - Large message handling
4. Fuzz testing - Parser robustness with Crowbar
5. Eio mock testing - Deterministic tests

Testing Strategy#

Unit Tests#

Each module should have corresponding tests in test/:

Integration Tests#

File: test/integration/

• Protocol compliance testing against real servers
• ImapTest compatibility suite
• Dovecot interoperability

Fuzz Tests#

File: test/fuzz_parser.ml

let fuzz_command_parser =
  Crowbar.(map [bytes] (fun input ->
    try
      ignore (Imap_parser.parse_command input);
      true
    with _ -> true  (* Parser should never crash *)
  ))

Implementation Order#

Sprint 1: P0 Critical Fixes#

1. Fix SEARCH response parsing
2. Parse BODY/BODYSTRUCTURE responses
3. Parse BODY[section] literals

Sprint 2: P1 Core Compliance#

4. Complete ESEARCH support
5. Parse APPENDUID/COPYUID response codes
6. Add UNSELECT to capabilities
7. Complete SPECIAL-USE support

Sprint 3: P2 SORT/THREAD#

8. Thread module types
9. Base subject extraction
10. Sent date handling
11. ORDEREDSUBJECT algorithm
12. REFERENCES algorithm
13. Server SORT/THREAD handlers

Sprint 4: P2 QUOTA#

14. Quota protocol types
15. Storage backend interface
16. Memory storage quota
17. Maildir storage quota
18. Server handlers

Sprint 5: P2 LIST-EXTENDED#

19. Extended LIST grammar
20. New attributes
21. Subscription tracking
22. RECURSIVEMATCH support

Sprint 6: P3 UTF-8 & CONDSTORE#

23. UTF-8 session state
24. UTF-8 validation
25. UTF8 APPEND extension
26. CONDSTORE types
27. CONDSTORE handlers
28. QRESYNC support

Sprint 7: P4 Polish#

29. Response code documentation
30. Unified mail flag library
31. Infrastructure improvements
32. Comprehensive test suite

File Modification Summary#

New Files#

Modified Files#

Design Principles#

1. Favor OCaml variants - Use typed variants over strings where possible
2. No backwards compatibility - Clean API without legacy shims
3. RFC citations - OCamldoc links to RFC sections
4. Incremental - Each task is independently useful
5. Test-driven - Tests accompany each feature
6. Eio-native - Use Eio patterns throughout

References#

Implemented RFCs#

• RFC 9051 - IMAP4rev2 (core)
• RFC 8314 - Implicit TLS
• RFC 2177 - IDLE
• RFC 2342 - NAMESPACE
• RFC 2971 - ID
• RFC 4315 - UIDPLUS
• RFC 5161 - ENABLE
• RFC 6851 - MOVE
• RFC 7888 - LITERAL+

RFCs in This Plan#

• RFC 3691 - UNSELECT (partially complete)
• RFC 4731 - ESEARCH
• RFC 5256 - SORT/THREAD
• RFC 5258 - LIST-EXTENDED
• RFC 5530 - Response Codes (types complete)
• RFC 6154 - SPECIAL-USE (partially complete)
• RFC 6855 - UTF-8 Support
• RFC 7162 - CONDSTORE/QRESYNC
• RFC 9208 - QUOTA