feat: add locking mechanism to batch task worker
Some checks failed
Docker Dev / docker (push) Has been cancelled
CI / scan_ruby (push) Has been cancelled
CI / lint (push) Has been cancelled
CI / test (push) Has been cancelled
Docker Main / docker (push) Has been cancelled

- Introduce a Redis-based lock to prevent concurrent execution of
  batch generation tasks.
- Set a TTL of 300 seconds for the lock to ensure it is released
  after a timeout.
- Add logging for situations where a task is already in progress.

This enhancement ensures that batch tasks do not overlap, which can
lead to data inconsistencies and resource contention. The locking
mechanism improves the reliability of the batch processing system.
This commit is contained in:
songtianlun 2025-02-22 15:41:42 +08:00
parent d331a73a85
commit 0e476b546d
2 changed files with 20 additions and 3 deletions

View File

@ -8,6 +8,25 @@ class BatchGenerateWeatherArtsWorker
PER_RUN_GENERATION_LIMIT = 2 # 每次运行生成图片上限
def perform(*args)
lock_key = "batch_generate_weather_lock"
lock_ttl = 300 # 锁的生存时间,单位为秒
redis = Redis.new(url: ENV.fetch("REDIS_URL", "redis://localhost:6379/1"))
if redis.set(lock_key, Time.current.to_s, nx: true, ex: lock_ttl)
begin
batch_tasks
ensure
redis.del(lock_key)
end
else
Rails.logger.info "Sitemap refresh is already in progress"
end
end
private
def batch_tasks
start_time = Time.current
remaining_slots = calculate_remaining_slots
return if remaining_slots <= 0
@ -38,8 +57,6 @@ class BatchGenerateWeatherArtsWorker
print_summary(processed_cities, skipped_cities)
end
private
def get_recent_cities
cutoff_time = Time.current - GENERATION_INTERVAL
City.joins("LEFT JOIN (

View File

@ -32,7 +32,7 @@ namespace :geo do
region.update!(
name: data["name"],
code: data["name"],
translations: data["translations"].to_json,
translations: data["translations"].to_json.to_s,
flag: data["flag"] || true,
wiki_data_id: data["wikiDataId"]
)