apm-server

apm-server copied to clipboard

Let's move this public document to the documentation to improve discovery, introduce versioning, and improve version control.

I propose adding these to the data model documentation but open to suggestions.

@graphaelli Is this still something we want to do?

Yes, this is still important.

Some thoughts and questions.

Is this Google Sheet up to date? It looks like it hasn't been updated in a while.

What's the relationship between this sheet and the span_types.json spec? The spec seems to be more up to date, but doesn't mention the span.name pattern/example or span.action.

If the Google Sheet and span_types.json spec are related, it seems like we already have two (public) sources of truth for this information. Adding asciidoc documentation would only add another source.

What's important for the user to know? We could easily generate documentation from the span_types.json spec; a rough table like this, for example:

Click to expand a very bad table that was generated

Description of span.type	span.type	span.subtype	Description
Spans within the application (usually not calling an external system	app	inferred	Sampling profiler inferred spans
	app	internal	Application generic internal span for controller/handler/processing delegation
API custom instrumentation	custom		API custom instrumentation
database span	db	cassandra	Cassandra
	db	cosmosdb	Azure CosmosDB
	db	db2	IBM DB2
	db	derby	Apache Derby
	db	dynamodb	AWS DynamoDB
	db	elasticsearch	Ealsticsearch
	db	graphql	GraphQL
	db	h2	H2
	db	hsqldb	HSQLDB
	db	ingres	Ingres
	db	mariadb	Maria DB
	db	memcached	Memcached
	db	mongodb	MongoDB
	db	mssql	Microsoft SQL Server
	db	mysql	MySQL
	db	oracle	Oracle Database
	db	postgresql	PostgreSQL
	db	redis	Redis
	db	sqlite	SQLite
	db	unkown	Unkown database
Request to external service, usually in request/response pattern	external	dubbo	Apache Dubbo
	external	grpc	gRPC
	external	http	HTTP client
	external	ldap	LDAP client
Messaging	messaging	azurequeue	Azure Queue
	messaging	azureservicebus	Azure Service Bus
	messaging	jms	Java Messaging Service
	messaging	kafka	Apache Kafka
	messaging	rabbitmq	RabbitMQ
	messaging	sns	AWS Simple Notification Service
	messaging	sqs	AWS Simple Queue Service
External process	process
	storage	azureblob	Azure Blob Storage
	storage	azurefile	Azure Files
	storage	azuretable	Azure Storage Table
	storage	s3	AWS S3
Template engines (no sub-type for now as really platform-specific)	template
Websockets	websocket	send

But if the span.action and span.name are important to document, does the spreadsheet need to be updated to include those for all available types/subtypes listed in the spec?

Or, am I overthinking all of this, i.e. the spec isn't important or related, and only the fields in the google sheet are important?

cc @alexanderwert

Great points @bmorelli25 . I believe we should get rid of the spreadsheet and instead reflect the missing parts in the spec directly and then use this as the source of truth.

👋 Picking this up again. I'm still trying to tease out why we want to publish this spec and which use cases would bring a user to this document. Gil initially mentioned these three goals for moving the span name/type spreadsheet to our docs (emphasis mine).

Let's move this public document to the documentation to improve discovery, introduce versioning, and improve version control.

It sounds to me like the versioning and version control problems are solved by moving from the spreadsheet to the spec — which leaves improved discovery. But I reached out the @felixbarny on Slack about publishing the spec, and he mentioned that only a few agents actually test against the spec.

So, do we want to bring attention to the spec if only a few agents use it? And if so, can someone help me understand the why?

The four potential outcomes from this issue I see are:

1. Document the spec as-is, explaining that only certain agents test against it.
2. Update the spec to include fields and examples from the spreadsheet, then doc it and explain that only certain agents test against it.
3. Don't update anything. Link to the spec and/or spreadsheet from the data-model/spans page and briefly explain what they're useful for.
4. Don't doc anything.

cc @SylvainJuge who originally created the spec.

I like suggestion (3); with the addition to completely retire the spreadsheet and ensure everything that is still valid is in the spec. WDYT?

I would also be in favor of retiring the spreadsheet.

Currently the "executable spec" in JSON only covers the span type/subtype, retiring the spreadsheet completely means quite a lot:

• adding the "action" field to the spans spec (I don't know exactly how and if it's actually used in the product, so it might not be worth the effort)
• creating an equivalent spec for transactions, should at least cover transaction type
• adding a few examples for transaction and spans naming, that would only be provided for documentation as it would be quite complex to enforce those (for example link between "execute a query on Redis" and "expect the redis span to have this name" might be platform-specific and API dependant.

The JSON document currently documents some inconsistencies, for example with the Ruby agent (see https://github.com/elastic/apm-agent-ruby/pull/1183 for details). While not all agents enforce this specification, we can still point to it and use it for documentation purposes.

Also, a while ago we started thinking about name alignment, see https://github.com/elastic/apm/issues/446 for details.

On Wed, Apr 27, 2022 at 8:57 AM Silvia Mitter @.***> wrote:

I like suggestion (3); with the addition to completely retire the spreadsheet and ensure everything that is still valid is in the spec. WDYT?

— Reply to this email directly, view it on GitHub https://github.com/elastic/apm-server/issues/3569#issuecomment-1110616430, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAF2JSR4TJKRE3KAO5TE2YTVHDQO5ANCNFSM4LVSVXWA . You are receiving this because you were mentioned.Message ID: @.***>

Thanks! So it sounds like the path forward could look something like this:

1. Minimal docs

• [x] Link to the spec and spreadsheet from the data-model/spans page and briefly explain what each is useful for. Done in https://github.com/elastic/apm-server/pull/8056.

2. Retire the spreadsheet

• [ ] Add the span.action field to the json spec (if necessary)
• [ ] Add example transaction/span names to the spec
• [ ] Create a transaction spec that covers transaction type

3. Update the docs

• [ ] Add the transaction and span specs to the documentation