feat: optimize batch weather art generation
- Introduce BATCH_SIZE constant to limit processed cities - Shuffle and limit eligible cities processing to enhance worker efficiency This update improves the performance of the BatchGenerateWeatherArtsWorker by ensuring that only a set number of cities are processed within the allotted time, reducing the risk of timeouts and making the overall system more responsive.
This commit is contained in:
parent
9417358625
commit
f2951e2741
@ -1,19 +1,16 @@
|
||||
class BatchGenerateWeatherArtsWorker
|
||||
include Sidekiq::Worker
|
||||
|
||||
GENERATION_INTERVAL = 24.hours
|
||||
MAX_DURATION = 50.minutes
|
||||
SLEEP_DURATION = 120.seconds
|
||||
BATCH_SIZE = 20
|
||||
|
||||
def perform(*args)
|
||||
start_time = Time.current
|
||||
|
||||
cities_to_process = get_eligible_cities
|
||||
|
||||
cities_to_process = get_eligible_cities.shuffle.take(BATCH_SIZE)
|
||||
cities_to_process.each do |city|
|
||||
break if Time.current - start_time > MAX_DURATION
|
||||
Rails.logger.info "Generating weather art for #{city.name}"
|
||||
|
||||
GenerateWeatherArtWorker.perform_async(city.id)
|
||||
sleep SLEEP_DURATION
|
||||
end
|
||||
@ -23,7 +20,6 @@ class BatchGenerateWeatherArtsWorker
|
||||
|
||||
def get_eligible_cities
|
||||
cutoff_time = Time.current - GENERATION_INTERVAL
|
||||
|
||||
City.active
|
||||
.joins("LEFT JOIN (
|
||||
SELECT city_id, MAX(created_at) as last_generation_time
|
||||
|
Loading…
Reference in New Issue
Block a user