feat: add locking mechanism to batch task worker
- Introduce a Redis-based lock to prevent concurrent execution of batch generation tasks. - Set a TTL of 300 seconds for the lock to ensure it is released after a timeout. - Add logging for situations where a task is already in progress. This enhancement ensures that batch tasks do not overlap, which can lead to data inconsistencies and resource contention. The locking mechanism improves the reliability of the batch processing system.
This commit is contained in:
parent
d331a73a85
commit
0e476b546d
@ -8,6 +8,25 @@ class BatchGenerateWeatherArtsWorker
|
||||
PER_RUN_GENERATION_LIMIT = 2 # 每次运行生成图片上限
|
||||
|
||||
def perform(*args)
|
||||
lock_key = "batch_generate_weather_lock"
|
||||
lock_ttl = 300 # 锁的生存时间,单位为秒
|
||||
|
||||
redis = Redis.new(url: ENV.fetch("REDIS_URL", "redis://localhost:6379/1"))
|
||||
|
||||
if redis.set(lock_key, Time.current.to_s, nx: true, ex: lock_ttl)
|
||||
begin
|
||||
batch_tasks
|
||||
ensure
|
||||
redis.del(lock_key)
|
||||
end
|
||||
else
|
||||
Rails.logger.info "Sitemap refresh is already in progress"
|
||||
end
|
||||
end
|
||||
|
||||
private
|
||||
|
||||
def batch_tasks
|
||||
start_time = Time.current
|
||||
remaining_slots = calculate_remaining_slots
|
||||
return if remaining_slots <= 0
|
||||
@ -38,8 +57,6 @@ class BatchGenerateWeatherArtsWorker
|
||||
print_summary(processed_cities, skipped_cities)
|
||||
end
|
||||
|
||||
private
|
||||
|
||||
def get_recent_cities
|
||||
cutoff_time = Time.current - GENERATION_INTERVAL
|
||||
City.joins("LEFT JOIN (
|
||||
|
@ -32,7 +32,7 @@ namespace :geo do
|
||||
region.update!(
|
||||
name: data["name"],
|
||||
code: data["name"],
|
||||
translations: data["translations"].to_json,
|
||||
translations: data["translations"].to_json.to_s,
|
||||
flag: data["flag"] || true,
|
||||
wiki_data_id: data["wikiDataId"]
|
||||
)
|
||||
|
Loading…
Reference in New Issue
Block a user