Separate service and framework code

[banool]

Note to the reviewer: Before I go and add all the comments and do some clean up I want to get eyes on this to make sure yall think the structure looks good.

Summary

This PR organizes the code in rust/ into three directories:

• framework/
  • This is the code that devs import in order to write processors.
• examples/
  • This contains examples of using the framework to write processors.
• service/
  • This contains our actual implementation of our processor stack.

The most notable crate within framework is aptos-processor-framework. This serves as a top level crate that users import and it gives them everything they need to subscribe the txn stream and push txns into a channel, pull things off that channel and dispatch them to processors, write a processor, deal with storage, etc. It also re-exports other framework crates like the protos and the currently-empty aptos-txn-parsers crate.

For now this PR doesn't make the code in service/ actually use the framework, I want to do that in a later PR otherwise this one will become way too big. So there will be some code dupe for a second.

Mostly I'm just moving code around but some notable refactoring / changes I made include:

• The core logic in worker.rs has been split into two parts:
  • stream_subscriber: This subscribes to the grpc stream and pushes txns into the channel.
  • dispatcher: This pulls txns off the channel and hands them to the processor.
  • This split gives the caller more flexibility with how they schedule the work, allows us to more easily create different stream subscriber implementations / mock out that component for testing, and because they were always two separate things to begin with.
• The tight coupling with diesel + postgres has been broken. The core logic in worker.rs only needed storage for the chain ID and the latest processed version. Instead these other components now operate something that implements a StorageTrait. This way the dev can choose whatever storage system they want.
  • We can provide concrete storage implementations down the line. I'd like us to provide more opinionated pieces, but it just needs to be separate from the core framework stuff.
• All the counter pushing stuff has been gated behind a feature. Many users won't want any of that.

Other changes:

• I add a .dockerignore file.

Worth calling out is I renamed processor to aptos-processor-service. I assume this will require changes to how we create containers from the image.

Test Plan

cd rust
docker build -t aptos-processor-service .

cd rust
cargo build --all

[banool]

I'm not sure what to make of this comment @bowenyang007, the framework is the main thing in this PR? Specifically which parts do you think we should take out of this PR.

[bowenyang007]

I thought you are renaming things, adding an example, and then refactoring the code. I think that the refactor needs discussion.

[banool]

Not much renaming happening without the refactor, otherwise all there is is two crates, and I can't add an example without the refactor either, otherwise there is nothing to easily import and use. Point being there's nothing to land here without the refactor, so we should just discuss the refactor and work on it in this PR.

Let's take this offline, we can all dive into it.

[banool]

Putting this in draft, we want to drive this with a design doc first.