questdb
questdb copied to clipboard
Published
20 hours ago •
questdb
questdb
refactor(core): Refactor ServerMain so that it may be instantiated from a different main block
Overview
With this refactor QuestDB may be instantiated in one of two forms:
- The boolean flag causes QuestDB to register a shutdown hook, which calls close:
new ServerMain("-d", "root-path").start(true) - Close is called at the end of the statement:
try (ServerMain questdb = new ServerMain("-d", "root-path")) { questdb.start(); }
The startup process of QuestDB is divided in three phases:
- Bootstrap: sets up logger/validates the configuration/checks the file system (purges old partitions and files)/sets up metrics/extracts the site.
- Services setup: including pg-wire, http, min-http, ilp-udp, ilp-tcp, telemetry, snapshot agent and others. Ports are bound now, worker pools are instantiated and registered.
- Start: worker pools are started and QuestDB is ready to receive SQL.
Implementation
There are a couple new classes: Services and WorkerPoolManager.
- Services: Is a factory user in startup phase 2 (Services setup). It makes use of the WorkerPoolManager to configure the required WorkerPools.
-
WorkerPoolManager: Is a helper class and single entry point for registering WorkerPools across the codebase. When
getInstanceis called, a WorkerPool is returned, whether it be the shared pool instance, or a newly created one. Instances are stored in a hash map indexed by pool name, which is now compulsory. All pools are started/closed with methodsstartAllandcloseAll.
this PR may have a merge fest with https://github.com/questdb/questdb/pull/2495
This is more significant refactoring that I anticipated. We need to write tests ensuring worker pool configuration works as expected. Right now it does not. I start server using the following config:
shared.worker.count=1
#http.worker.count=0
#shared.worker.affinity=2,4,6,8,10,12,14,16,18,20,22,24
#cairo.writer.fo_opts=o_direct
http.security.readonly=false
http.allow.deflate.before.send=false
#cairo.o3.max.uncommitted.rows=290000
#cairo.o3.commit.hysteresis.in.ms=300000
#line.tcp.maintenance.job.hysteresis.in.ms=30000
#line.tcp.min.idle.ms.before.writer.release=32000
#line.tcp.msg.buffer.size=32768
#line.tcp.n.updates.per.load.balance=10000000
#line.tcp.net.bind.to=10.0.0.1:9009
#line.tcp.max.uncommitted.rows=1000000
#line.tcp.maintenance.job.hysteresis.in.ms=15000
#line.tcp.min.idle.ms.before.writer.release=5000
#line.tcp.commit.hysteresis.in.ms=1800000
line.tcp.net.bind.to=0.0.0.0:9009
#line.tcp.auth.db.path=conf/authDb.txt
#line.tcp.writer.worker.count=0
#line.tcp.writer.worker.affinity=26,28
#line.tcp.writer.halt.on.error=true
#line.tcp.io.worker.count=0
#line.tcp.io.worker.affinity=30,32,34,36,38,40,42,44,46,48,50,52
#line.tcp.io.halt.on.error=true
line.tcp.net.io.queue.capacity=128
#line.tcp.max.measurement.size=2048
line.tcp.net.connection.rcvbuf=4M
http.bind.to=0.0.0.0:9000
line.udp.bind.to=0.0.0.0:9009
#pg.net.bind.to=0.0.0.0:5432
#pg.user=xyz
#pg.password=oh
telemetry.enabled=false
#cairo.sql.map.page.size=33554432
cairo.sql.hash.join.light.value.page.size=33554432
metrics.enabled=true
cairo.sql.jit.mode=on
cairo.sql.parallel.filter.enabled=true
cairo.sql.copy.root=C:/Users/Vlad/Downloads/
cairo.sql.copy.work.root=c:/qdb2/
#cairo.rss.memory.limit.gib=8
cairo.enable.crash.simulation=true
These are the pools server starts with:
2022-09-12T13:34:27.467137Z A server-main Bootstrap complete, ready to start
2022-09-12T13:34:27.467790Z I i.q.m.WorkerPoolManager Started shared pool [name=worker, workers=1]
2022-09-12T13:34:27.467888Z I i.q.m.WorkerPoolManager Started dedicated pool [name=minhttp, workers=1]
2022-09-12T13:34:27.468067Z I server-main os scheduled worker started [name=questdb-worker-2]
2022-09-12T13:34:27.468141Z I server-main os scheduled worker started [name=questdb-minhttp-3]
2022-09-12T13:34:27.468241Z I server-main os scheduled worker started [name=questdb-ilpio-4]
2022-09-12T13:34:27.468274Z I server-main os scheduled worker started [name=questdb-ilpio-5]
2022-09-12T13:34:27.468328Z I i.q.m.WorkerPoolManager Started dedicated pool [name=ilpio, workers=6]
2022-09-12T13:34:27.468413Z I server-main os scheduled worker started [name=questdb-ilpio-7]
2022-09-12T13:34:27.468467Z I server-main os scheduled worker started [name=questdb-ilpio-6]
2022-09-12T13:34:27.468468Z I i.q.m.WorkerPoolManager Started dedicated pool [name=ilpwriter, workers=1]
2022-09-12T13:34:27.468490Z I server-main
___ _ ____ ____
/ _ \ _ _ ___ ___| |_| _ \| __ )
| | | | | | |/ _ \/ __| __| | | | _ \
| |_| | |_| | __/\__ \ |_| |_| | |_) |
\__\_\\__,_|\___||___/\__|____/|____/
web console URL(s):
http:///127.0.0.1:9000
http:///192.168.1.218:9000
http:///172.26.0.1:9000
http:///172.26.96.1:9000
2022-09-12T13:34:27.468517Z I server-main os scheduled worker started [name=questdb-ilpio-9]
2022-09-12T13:34:27.468585Z I server-main os scheduled worker started [name=questdb-ilpwriter-10]
2022-09-12T13:34:27.504983Z A server-main enjoy
What is wrong with the above? I didn't configure dedicated pool for "min" server and it started one. I didn't configure dedicated pools for ILP - it started a lot. Master ILP is also broken, although "min" works as expected:
2022-09-12T13:38:07.745623Z A http-min-server listening on 0.0.0.0:9003 [fd=2436 backlog=4]
2022-09-12T13:38:07.753912Z I server-main started
2022-09-12T13:38:07.754103Z I server-main os scheduled [name=questdb-worker-2]
2022-09-12T13:38:07.757426Z A pg-server listening on 0.0.0.0:8812 [fd=2456 backlog=64]
2022-09-12T13:38:07.813966Z I i.q.c.l.u.AbstractLineProtoUdpReceiver receiving multicast from 232.1.2.3:9009 via 0.0.0.0 [fd=2464, commitRate=1000000]
2022-09-12T13:38:07.815040Z I i.q.c.l.t.LineTcpReceiver using default context
2022-09-12T13:38:07.815716Z A tcp-line-server listening on 0.0.0.0:9009 [fd=2476 backlog=256]
2022-09-12T13:38:07.871320Z I server-main os scheduled [name=questdb-ilpio-3]
2022-09-12T13:38:07.871407Z I server-main os scheduled [name=questdb-ilpio-4]
2022-09-12T13:38:07.871436Z I server-main started
2022-09-12T13:38:07.871514Z I server-main os scheduled [name=questdb-ilpio-5]
2022-09-12T13:38:07.871529Z I server-main os scheduled [name=questdb-ilpio-6]
2022-09-12T13:38:07.871594Z I server-main os scheduled [name=questdb-ilpio-7]
2022-09-12T13:38:07.871623Z I server-main started
2022-09-12T13:38:07.871650Z I server-main os scheduled [name=questdb-ilpio-8]
2022-09-12T13:38:07.871729Z I server-main os scheduled [name=questdb-ilpwriter-9]
2022-09-12T13:38:07.871770Z I server-main started
2022-09-12T13:38:07.871798Z I server-main web console URL(s):
One of the reasons we do this refactoring is to ensure Server behaviour does not regress
the docker build commands above are very useful, thank you, I was not entirely aware of all the switches:
docker buildx build --push --platform linux/arm64 --tag questdb/questdb:6.5.4-SNAPSHOT --build-arg tag_name=ma/server-main-instantiable .
[PR Coverage check]
:heart_eyes: pass : 774 / 897 (86.29%)
file detail
| path | covered line | new line | coverage | |
|---|---|---|---|---|
| :large_blue_circle: | io/questdb/network/IODispatcherWindows.java | 0 | 4 | 00.00% |
| :large_blue_circle: | io/questdb/network/MutableIOContextFactory.java | 0 | 1 | 00.00% |
| :large_blue_circle: | io/questdb/network/Kqueue.java | 0 | 4 | 00.00% |
| :large_blue_circle: | io/questdb/cutlass/http/DefaultHttpServerConfiguration.java | 0 | 2 | 00.00% |
| :large_blue_circle: | io/questdb/cutlass/line/tcp/DefaultLineTcpReceiverConfiguration.java | 1 | 5 | 20.00% |
| :large_blue_circle: | io/questdb/cutlass/line/tcp/LineTcpWriterJob.java | 1 | 2 | 50.00% |
| :large_blue_circle: | io/questdb/cutlass/http/processors/JsonQueryProcessor.java | 1 | 2 | 50.00% |
| :large_blue_circle: | io/questdb/cutlass/http/processors/TextQueryProcessor.java | 1 | 2 | 50.00% |
| :large_blue_circle: | io/questdb/mp/Worker.java | 5 | 7 | 71.43% |
| :large_blue_circle: | io/questdb/ServerMain.java | 64 | 84 | 76.19% |
| :large_blue_circle: | io/questdb/cutlass/pgwire/PGWireServer.java | 4 | 5 | 80.00% |
| :large_blue_circle: | io/questdb/cutlass/line/udp/AbstractLineProtoUdpReceiver.java | 4 | 5 | 80.00% |
| :large_blue_circle: | io/questdb/cutlass/Services.java | 44 | 54 | 81.48% |
| :large_blue_circle: | io/questdb/Bootstrap.java | 227 | 276 | 82.25% |
| :large_blue_circle: | io/questdb/log/LogFactory.java | 136 | 154 | 88.31% |
| :large_blue_circle: | io/questdb/network/AbstractIODispatcher.java | 38 | 41 | 92.68% |
| :large_blue_circle: | io/questdb/mp/WorkerPool.java | 36 | 37 | 97.30% |
| :large_blue_circle: | io/questdb/griffin/engine/functions/test/TestSumTDoubleGroupByFunction.java | 2 | 2 | 100.00% |
| :large_blue_circle: | io/questdb/std/str/Path.java | 1 | 1 | 100.00% |
| :large_blue_circle: | io/questdb/cutlass/line/tcp/LineTcpMeasurementScheduler.java | 2 | 2 | 100.00% |
| :large_blue_circle: | io/questdb/griffin/engine/groupby/vect/AvgDoubleVectorAggregateFunction.java | 2 | 2 | 100.00% |
| :large_blue_circle: | io/questdb/griffin/engine/ops/AbstractOperation.java | 1 | 1 | 100.00% |
| :large_blue_circle: | io/questdb/PropServerConfiguration.java | 10 | 10 | 100.00% |
| :large_blue_circle: | io/questdb/cairo/O3Utils.java | 4 | 4 | 100.00% |
| :large_blue_circle: | io/questdb/cairo/mig/MigrationContext.java | 1 | 1 | 100.00% |
| :large_blue_circle: | io/questdb/std/Rosti.java | 2 | 2 | 100.00% |
| :large_blue_circle: | io/questdb/cairo/TableReaderMetadata.java | 2 | 2 | 100.00% |
| :large_blue_circle: | io/questdb/WorkerPoolManager.java | 67 | 67 | 100.00% |
| :large_blue_circle: | io/questdb/Metrics.java | 1 | 1 | 100.00% |
| :large_blue_circle: | io/questdb/log/LogRollingFileWriter.java | 2 | 2 | 100.00% |
| :large_blue_circle: | io/questdb/griffin/engine/functions/catalogue/AbstractPgClassFunctionFactory.java | 2 | 2 | 100.00% |
| :large_blue_circle: | io/questdb/griffin/engine/groupby/SampleByInterpolateRecordCursorFactory.java | 2 | 2 | 100.00% |
| :large_blue_circle: | io/questdb/cutlass/line/udp/LineUdpLexer.java | 3 | 3 | 100.00% |
| :large_blue_circle: | io/questdb/log/LogAlertSocket.java | 4 | 4 | 100.00% |
| :large_blue_circle: | io/questdb/cutlass/http/processors/HealthCheckProcessor.java | 1 | 1 | 100.00% |
| :large_blue_circle: | io/questdb/cairo/mig/Mig620.java | 1 | 1 | 100.00% |
| :large_blue_circle: | io/questdb/cutlass/line/tcp/LineTcpReceiver.java | 1 | 1 | 100.00% |
| :large_blue_circle: | io/questdb/cutlass/text/AtomicBooleanCircuitBreaker.java | 1 | 1 | 100.00% |
| :large_blue_circle: | io/questdb/std/str/DirectCharSink.java | 3 | 3 | 100.00% |
| :large_blue_circle: | io/questdb/cutlass/json/JsonLexer.java | 4 | 4 | 100.00% |
| :large_blue_circle: | io/questdb/mp/WorkerPoolConfiguration.java | 1 | 1 | 100.00% |
| :large_blue_circle: | io/questdb/network/Epoll.java | 2 | 2 | 100.00% |
| :large_blue_circle: | io/questdb/cutlass/http/HttpServer.java | 9 | 9 | 100.00% |
| :large_blue_circle: | io/questdb/cairo/mig/Mig608.java | 1 | 1 | 100.00% |
| :large_blue_circle: | io/questdb/cutlass/text/TextLexer.java | 4 | 4 | 100.00% |
| :large_blue_circle: | io/questdb/cutlass/line/tcp/LineTcpConnectionContext.java | 3 | 3 | 100.00% |
| :large_blue_circle: | io/questdb/cairo/mig/Mig605.java | 1 | 1 | 100.00% |
| :large_blue_circle: | io/questdb/cutlass/line/udp/LineUdpReceiver.java | 2 | 2 | 100.00% |
| :large_blue_circle: | io/questdb/cutlass/line/AbstractLineSender.java | 4 | 4 | 100.00% |
| :large_blue_circle: | io/questdb/cutlass/http/processors/TextQueryProcessorState.java | 1 | 1 | 100.00% |
| :large_blue_circle: | io/questdb/griffin/engine/functions/rnd/RndStringMemory.java | 2 | 2 | 100.00% |
| :large_blue_circle: | io/questdb/griffin/engine/groupby/vect/AvgLongVectorAggregateFunction.java | 2 | 2 | 100.00% |
| :large_blue_circle: | io/questdb/griffin/engine/table/LatestByArguments.java | 4 | 4 | 100.00% |
| :large_blue_circle: | io/questdb/griffin/engine/functions/test/TestSumStringGroupByFunction.java | 2 | 2 | 100.00% |
| :large_blue_circle: | io/questdb/cutlass/line/tcp/DelegatingTlsChannel.java | 5 | 5 | 100.00% |
| :large_blue_circle: | io/questdb/cairo/CairoEngine.java | 6 | 6 | 100.00% |
| :large_blue_circle: | io/questdb/cairo/mig/Mig607.java | 1 | 1 | 100.00% |
| :large_blue_circle: | io/questdb/cutlass/text/types/InputFormatConfiguration.java | 2 | 2 | 100.00% |
| :large_blue_circle: | io/questdb/griffin/engine/groupby/vect/AvgIntVectorAggregateFunction.java | 2 | 2 | 100.00% |
| :large_blue_circle: | io/questdb/log/LogFileWriter.java | 2 | 2 | 100.00% |
| :large_blue_circle: | io/questdb/cutlass/pgwire/DefaultPGWireConfiguration.java | 1 | 1 | 100.00% |
| :large_blue_circle: | io/questdb/std/MemoryTag.java | 10 | 10 | 100.00% |
| :large_blue_circle: | io/questdb/cutlass/text/TextDelimiterScanner.java | 2 | 2 | 100.00% |
| :large_blue_circle: | io/questdb/cairo/ColumnPurgeJob.java | 2 | 2 | 100.00% |
| :large_blue_circle: | io/questdb/cairo/vm/MemoryPARWImpl.java | 1 | 1 | 100.00% |
| :large_blue_circle: | io/questdb/griffin/SqlCompiler.java | 3 | 3 | 100.00% |
| :large_blue_circle: | io/questdb/cairo/mig/Mig609.java | 1 | 1 | 100.00% |
| :large_blue_circle: | io/questdb/griffin/engine/functions/catalogue/PgAttrDefFunctionFactory.java | 2 | 2 | 100.00% |
| :large_blue_circle: | io/questdb/cutlass/http/processors/QueryCache.java | 4 | 4 | 100.00% |
| :large_blue_circle: | io/questdb/cairo/mig/EngineMigration.java | 3 | 3 | 100.00% |
| :large_blue_circle: | io/questdb/metrics/HealthMetricsImpl.java | 1 | 1 | 100.00% |
| :large_blue_circle: | io/questdb/cutlass/text/TextMetadataParser.java | 3 | 3 | 100.00% |
| :large_blue_circle: | io/questdb/cutlass/http/processors/JsonQueryProcessorState.java | 1 | 1 | 100.00% |
| :large_blue_circle: | io/questdb/cairo/ColumnPurgeOperator.java | 2 | 2 | 100.00% |
| :large_blue_circle: | io/questdb/cutlass/text/TextImportJob.java | 1 | 1 | 100.00% |
