support for concurrency?

[SUP3RIA]

Its not really mentioned in your documentation but your code does not seem to support concurrency? I tried to login and fetch emails from 10000 goroutines at the same time and it is blocking after a while.

Would it not be possible to support concurrency by using sync.Mutex? Its such a shame because its the best IMAP lib for Golang and the point of using Golang for the most people is the concurrency.

Here is my code:

// Connect to server
	c, err := client.DialTLS(domain+port, nil)
	if err != nil {
		if show_errors{
			fmt.Println(b[0],":", err)
		}
		return
	}
	defer c.Logout()
	// Login
	//color.Blue(r)
	if err := c.Login(name, password); err != nil {
		//dealing with gmail's response if imap is not activated by user
		if strings.Contains(err.Error() , "Please log in via your web browser:"){
			fmt.Println(">>True "+r+"|login via web only!")
			return
		}
		if err != nil {
			fmt.Println("<<False "+r)
		}
	} else {
		fmt.Println(">>True "+r)
		// List mailboxes
		mailboxes := make(chan *imap.MailboxInfo, 10)
		done := make(chan error, 1)
		go func() {
			done <- c.List("", "*", mailboxes)
		}()

		var list []*imap.MailboxInfo
		for m := range mailboxes {
			list = append(list, m)
		}

		if err := <-done; err != nil {
			log.Fatal(err)
		}
		for _, m := range list {
			mbox, err := c.Select(m.Name, true)
			if err != nil {
				log.Println("Error", err)
				return
			}
			if mbox.Messages>0{
			// Select INBOX
			_, err := c.Select(m.Name, true)
			if err != nil {
				log.Println(err)
			}
			criteria := &imap.SearchCriteria{

				Header: textproto.MIMEHeader{
					"From": {"gmail.com"},
				},

			}
			results, err := c.Search(criteria)
			if err != nil {
				log.Println(err)
			}

			if len(results)>0{
				seqset := new(imap.SeqSet)
				for i := range results {
					seqset.AddNum(uint32(i))
				}
				// Get the whole message body
				attrs := []string{"BODY[]"}
				messages := make(chan *imap.Message, 1)
				go func() {
					if err := c.Fetch(seqset, attrs, messages); err != nil {
						log.Fatal(err)
					}
				}()
				msg := <-messages
				r := msg.GetBody("BODY[]")
				if r == nil {
					log.Fatal("Server didn't returned message body")
				}

				// Create a new mail reader
				mr, err := mail.CreateReader(r)
				if err != nil {
					log.Fatal(err)
				}

				// Print some info about the message
				header := mr.Header
				if date, err := header.Date(); err == nil {
					log.Println("Date:", date)
				}
				if from, err := header.AddressList("From"); err == nil {
					log.Println("From:", from)
				}
				if subject, err := header.Subject(); err == nil {
					log.Println("Subject:", subject)
				}

				for {
					p, err := mr.NextPart()
					if err == io.EOF {
						break
					}
					b, _ := ioutil.ReadAll(p.Body)
					log.Println(string(b))
				}
			}

		    }
			c.Close()
	    }	
	}
	defer c.Logout()

[emersion]

First, there are issues with deadlocks when executing many commands in a loop, see https://github.com/emersion/go-imap/issues/107

Then, in my opinion the client shouldn't handle concurrency. Some commands are safe to execute concurrently (see the IMAP RFC) but I don't think this library should do something if you execute at the same time commands that can't be issued in parallel. Doing so would add a performance overhead for users who don't do it, and you shouldn't do it in the first place according to the RFC. This is somewhat the default behavior in the Go standard library too: the built-in map type is not safe to use from different goroutines, if you want to do so, you have to use a mutex yourself.

[emersion]

Now that #107 is fixed in the v1 branch, it should be possible to fetch messages from different goroutines.

[neatcode]

Go isn't good for concurrency actually. You have to manage everything with mutex's etc and you get locking issues.

[emersion]

Sorry, this issue isn't a place to promote My Beloved Language™. Removed reference to it.

[engineersaurav]

Hi @emersion , Could we seprate promotions and socials from INBOX in Gmail using your package. Like for now promotional and social emails comes under INBOX in gmail. Thanks

Regards Saurav Kumar

[tonimelisma]

Maybe this is the right issue to raise the question: why do List(), Fetch() and other functions work with channels when they're working on a linear IMAP connection? I assume there's no way to parallelize these commands on the IMAP connection, so actually even if I run several of these concurrently in Go, they will be executed on a single connection to the IMAP server, one after the other, right?

So what does the concurrency improve upon? The software can parallelize marshaling/unmarshaling the IMAP responses to the go data structures..? I would assume the constraint is always on the network side.

[emersion]

These use channels so that results can be handled as they come by the library user, and the library doesn't need to allocate a (potentially large) slice.

[emersion]

Done in v2.