|
|
f2e65fd6cd
|
hunter: split into butcher
|
2023-01-22 00:05:10 +01:00 |
|
|
|
077feb4c6c
|
hunter/trend_setter: fix removal of tags from trend_pools
|
2022-12-30 19:51:09 +01:00 |
|
|
|
33d5221e12
|
hunter/main: enable reqwest tcp_keepalive
|
2022-12-29 15:30:35 +01:00 |
|
|
|
df50718724
|
Revert "hunter/worker: invert hunter_new_post_ratio metric"
This reverts commit 5260090b19.
|
2022-12-29 03:17:03 +01:00 |
|
|
|
5260090b19
|
hunter/worker: invert hunter_new_post_ratio metric
|
2022-12-27 20:34:34 +01:00 |
|
|
|
704b883de2
|
hunter/worker: adjust new posts log level
|
2022-12-27 19:05:53 +01:00 |
|
|
|
ff9c64ec1a
|
hunter: add more metrics
|
2022-12-27 19:03:09 +01:00 |
|
|
|
01972899ae
|
hunter/trend_setter: reduce metrics histogram facets
|
2022-12-27 03:54:09 +01:00 |
|
|
|
feb60d9805
|
hunter/worker: measure hunter_worker_starts
|
2022-12-26 04:30:27 +01:00 |
|
|
|
262ac7ad73
|
hunter/trend_setter: add metrics
|
2022-12-26 04:21:42 +01:00 |
|
|
|
aeec4045a0
|
hunter/worker: catch runaway metrics
|
2022-12-26 04:14:10 +01:00 |
|
|
|
ce3f184d04
|
hunter: add metrics for prometheus
|
2022-12-26 03:46:11 +01:00 |
|
|
|
f5fb098bd2
|
switch from log to tracing
|
2022-12-26 03:05:00 +01:00 |
|
|
|
21e670cd2c
|
hunter: remove evil hosts from redis again
|
2022-12-26 02:49:43 +01:00 |
|
|
|
484992b92b
|
hunter/main: call systemd::extend_timeout() only every 1000 reintroduced hosts
|
2022-12-26 02:25:50 +01:00 |
|
|
|
0195cab965
|
hunter/scheduler: commit the fixed EVIL_DOMAINS check
|
2022-12-26 02:25:13 +01:00 |
|
|
|
47aa137c22
|
Cargo.toml: fix by moving optimization config to project root
|
2022-12-04 03:55:36 +01:00 |
|
|
|
ea0fe54106
|
hunter/scheduler: filter EVIL_DOMAINS
|
2022-12-04 03:54:42 +01:00 |
|
|
|
908fb9c8d3
|
*/Cargo.toml: add optimization flags
|
2022-12-03 01:51:38 +01:00 |
|
|
|
c1d2c3786d
|
hunter/main: use jemalloc
|
2022-12-03 01:47:44 +01:00 |
|
|
|
fed37ce56f
|
hunter/main: enable trust-dns
seems to use even more memory but perhaps we're no longer bound by
worker threads?
|
2022-12-03 01:47:07 +01:00 |
|
|
|
cd132a67bc
|
cave/feed: clean up with EncodedPost
|
2022-12-02 23:05:35 +01:00 |
|
|
|
f7ebb08274
|
cave/feed: wrap Post in EncodablePost
|
2022-12-02 22:03:38 +01:00 |
|
|
|
51a21d3fbc
|
hunter: Arcify host
|
2022-12-02 00:50:01 +01:00 |
|
|
|
69ba3ed657
|
posts_cache: use only 1 Arc
|
2022-12-02 00:30:10 +01:00 |
|
|
|
f03a83e9cd
|
hunter/tag_trimmer: act more gently
|
2022-11-24 19:25:03 +01:00 |
|
|
|
77997e9720
|
hunter: redisable reqwst feature rustls
|
2022-11-24 19:24:44 +01:00 |
|
|
|
00ded7dc8e
|
hunter: add posts_cache
|
2022-11-23 23:59:35 +01:00 |
|
|
|
91dcf74609
|
hunter/worker: free RobotsTxt earlier
|
2022-11-18 20:13:34 +01:00 |
|
|
|
3ccc6278d9
|
hunter/main: bump systemd::extend_timeout() for introducing hosts from redis
|
2022-11-18 17:43:35 +01:00 |
|
|
|
73b3641605
|
hunter/main: extend systemd StartTimeout
|
2022-11-15 19:02:14 +01:00 |
|
|
|
e622eace28
|
hunter/main: introduce hosts from redis only on production
|
2022-11-14 22:45:33 +01:00 |
|
|
|
4684274d6d
|
hunter/trend_setter: bump MIN_INTERVAL
|
2022-11-14 21:52:25 +01:00 |
|
|
|
d65abb82c3
|
hunter/main: reenable introducing hosts from redis
|
2022-11-14 19:30:19 +01:00 |
|
|
|
e49ae907e5
|
cave/trend_tag: switch from by_hours to hour_users algorithm
|
2022-11-14 03:14:12 +01:00 |
|
|
|
48156b640c
|
cave/store: bump max_pool_size
|
2022-11-12 02:47:21 +01:00 |
|
|
|
d35004c4d1
|
hunter/tag_timmer: add
|
2022-11-12 01:02:44 +01:00 |
|
|
|
663db47973
|
add working streaming
|
2022-11-11 22:13:33 +01:00 |
|
|
|
0e5fab858e
|
hunter/worker: eval robot.delay
|
2022-11-11 19:21:44 +01:00 |
|
|
|
53ae6e6ba1
|
hunter/worker: refactor, prepare stream, add robots.txt support
|
2022-11-11 19:00:37 +01:00 |
|
|
|
e620c69a32
|
hunter: refactor worker to prepare for more fetch steps
|
2022-11-11 18:08:13 +01:00 |
|
|
|
51a4f7c5ea
|
make languages depend on trend pool size
|
2022-11-11 17:11:13 +01:00 |
|
|
|
adedb40105
|
cave/store: add RedisPool
|
2022-11-10 15:22:45 +01:00 |
|
|
|
aa715aa377
|
hunter/trend_setter: keep score <= 0 out of pools
|
2022-11-10 03:28:51 +01:00 |
|
|
|
d42615b44e
|
hunter: fix debug output
|
2022-11-09 19:25:51 +01:00 |
|
|
|
1092194bba
|
move all trend filters from gatherer to hunter
|
2022-11-09 19:25:07 +01:00 |
|
|
|
e224c14c86
|
gatherer: adopt trend pools
|
2022-11-09 19:03:03 +01:00 |
|
|
|
0c23fad76d
|
hunter/trend_setter: implement maintaining the pools
|
2022-11-09 18:11:02 +01:00 |
|
|
|
a98a669086
|
hunter: begin trend_setter
|
2022-11-08 00:43:46 +01:00 |
|
|
|
a5a5a10d15
|
hunter/redis_store: move into cave::store
|
2022-11-07 22:07:07 +01:00 |
|
