- Update cm-script.yaml to use S3_HOST_BUCKET
- Modify daemonset.yaml to reference S3_HOST_BUCKET
- Change secret.yaml to include agz-host-bucket
This refactor enhances the clarity of the configuration by using a more descriptive variable name. It ensures that all references to the S3 host bucket are consistent across the configuration files.
- Add 'type' field to s3cmd configuration
- Ensure compatibility with S3 provider settings
This change updates the s3cmd configuration to include a 'type' field,
which is necessary for proper interaction with the specified S3 provider.
This adjustment helps in avoiding potential misconfigurations.
- Change the APK repository URL to use a mirror site
- This ensures better availability and potentially faster downloads
The update to the repository URL is intended to improve the
reliability of package installations in the daemonset's
configuration.
- Introduce S3_PROVIDER variable in cm-script.yaml
- Update s3cmd configuration to include provider
- Modify daemonset.yaml to support tencent-gz1 and tencent-sh1 in node affinity
These changes allow the backup system to utilize multiple S3 providers, enhancing its compatibility and deployment options across different cloud environments.
- Introduce S3_PROVIDER environment variable in daemonset.yaml
- Update secret.yaml to include provider information
This change allows the application to specify the S3 provider type, improving
flexibility in storage configuration. The new variable is sourced from the
existing s3-credentials secret, ensuring secure access to the provider
information.
- Create ConfigMaps for backup configuration and scripts
- Define Secrets for S3 credentials
- Implement Role and RoleBinding for access control
- Set up a DaemonSet for running backup containers
- Add a CronJob to schedule backups daily
This commit establishes a comprehensive backup solution within the Kubernetes cluster, allowing for automated backups of specified directories to S3 storage. It includes necessary configurations and scripts to ensure proper execution and notification of backup status.
- Change provisioner name in zgo-us1.yaml to match the correct
provisioner.
- Update node selector in multiple statefulset manifests to
ensure they target the correct nodes.
- Replace storage class name from nfs-zgo-us1 to local-vkus2
for better resource management.
These changes ensure that the application components are
correctly configured to use the appropriate storage and
node resources, improving deployment stability and
performance.
- Remove specific node selector for vkvm-us2
- Add region and datacenter labels for better scheduling
- Increase storage requests for database and event services
- Change storage class from local-vkus2 to nfs-zgo-us1
These changes improve the flexibility of the deployment by allowing
Kubernetes to schedule pods based on region and datacenter, and
ensure that the services have adequate storage resources for
their operations.
- Replace hardcoded hostname with region and dc labels
- Update storageClassName from local-vkus2 to nfs-zgo-us1
These changes improve the flexibility of the deployment by allowing
for better node selection based on region and data center, and
ensure that the storage class is correctly set for the environment.
This will help in managing resources more effectively across
different clusters.
- Introduce podAntiAffinity settings to avoid scheduling
multiple Pods with the same app: taw label on the same
node.
- This is a soft requirement, meaning that if there are
not enough nodes, Pods may still be scheduled on the
same node.
This change enhances the deployment strategy by improving
availability and fault tolerance of the taw application.
- Change the number of replicas from 1 to 3 in the
deployment configuration.
- This adjustment aims to improve load balancing and
availability of the application.
Increasing the replicas allows the application to handle
more traffic and provides redundancy in case of failures.
- Change kind from DaemonSet to Deployment
- Uncomment replicas setting to ensure proper scaling
This change corrects the resource type for the application
in the Kubernetes configuration, ensuring it is deployed as a
Deployment instead of a DaemonSet. This allows for better
management of replicas and scaling capabilities.
- Adjust the script to include logic for cleaning up old backups
- Added support for handling PostgreSQL data directories
- Ensure temporary directories are cleaned after use
This update improves the backup process by ensuring that old backups
are properly cleaned up to save storage space and enhance efficiency.
It also includes logic to handle specific cases for PostgreSQL
directories, providing a more robust backup operation.
- Introduce a sleep command before triggering backup for each pod
- This change prevents simultaneous execution of backup tasks
- Ensures system stability by spreading out resource usage during backups
- Introduce SOURCE_SIZE variable in cm-script.yaml
- Remove redundant SOURCE_SIZE calculation
This change calculates the size of the data directory
prior to initiating the backup process. It ensures that
the backup script has accurate information about the size
of the source data, enhancing the logging and monitoring
of backup activities.
- Added special handling for PostgreSQL data directories to ensure
proper backup of the `pg_wal` folder.
- Backup now distinguishes between standard and PostgreSQL specific
logic, improving the reliability of database backups.
- Implemented warnings suppression for unchanged files to prevent
clutter during backups.
This update improves the robustness of the backup process, ensuring
that PostgreSQL data is handled correctly while also maintaining
standard directory backup functionality, providing better clarity
and usability to users performing backups.
- Implement node affinity to prevent scheduling on vkvm-us2
- Update affinity section in daemonset.yaml
- Ensure that the DaemonSet runs only on specific nodes
This change introduces a node affinity rule to the DaemonSet configuration,
allowing it to avoid scheduling on nodes labeled with `kubernetes.io/hostname`
set to `vkvm-us2`. This helps to ensure resource allocation and performance
by restricting the DaemonSet to the desired nodes.
- Change TEMP_DIR to use a more structured temporary path
- Adjust rsync command to reflect the new directory structure
- Improve MSG_TEXT formatting for better clarity
- Add 'jq' to the dependencies for JSON processing
These changes address issues with the previous temporary directory location and enhance the output messages for a more informative backup success notification.
- Add `haut-gis-org.github.io`, `vue_vue2leaflet`, and `lobe-chat-1` to the list of skipped repositories.
- This change ensures that these repositories are not processed by the cron job, helping streamline operations and avoid unnecessary tasks.
- Include a webhook URL to send notifications after backups.
- Capture and log the duration and size of the backups.
- Create a new Kubernetes secret for storing the Feishu webhook URL.
- Enhance the backup script to notify users of backup success with details.
This change improves monitoring and user notification of backup events,
allowing for better awareness and response times in case of failure or
success of the backup processes.
- Create a new namespace for the backup system
- Implement a cron job for scheduled backups
- Add a daemon set to handle backup tasks across nodes
- Introduce necessary service accounts, roles, and role bindings
- Include environment variable handling and configuration via secrets and config maps
- Ensure triggering and execution workflow for backups is efficient
This commit establishes a new backup system that utilizes both a cron job and a daemon set to automate backups. It organizes the configurations and credentials needed for S3-compatible storage, allowing for seamless backup management across the specified nodes in the Kubernetes cluster.
- Create a cronjob to back up node1 data to node8
- Define schedule for daily backups at 3:00 AM
- Include error handling and notifications via Feishu
- Use SSH and rsync for secure and efficient data transfer
This commit introduces a new cronjob that automates the backup process
for node1 to node8, enabling easier management and recovery of data.
The setup includes necessary security measures and proper logging of backups,
ensuring smoother operation and notifications in case of failures.
- Remove specific hostname constraint from
nodeSelector for various components and add
region and datacenter keys.
This change allows more flexible scheduling of pods
across available nodes by removing the fixed
hostname requirement. It introduces the ability to
distribute workloads based on region and data center
while maintaining functional consistency across the
Kubernetes deployment.
- Changed storageClass from "local-vkus1" to "nfs-zgo-us1" for various components:
- api
- proxy
- ssrfProxy
- pluginDaemon
- postgresql
- redis
This update standardizes the storage class across multiple components,
potentially improving compatibility and performance with NFS-based storage.
It may help in better resource management and scalability.
- Change kind from Deployment to DaemonSet
- Remove previous node selector configuration
- Add tolerations for scheduling restrictions
- Introduce node affinity for specific hostnames
These changes facilitate deployment across all nodes instead of a single replica,
allowing for improved resource utilization and management in the
Kubernetes environment.
- Bump image version for `chatgpt-web-midjourney-proxy` from 2.22.7 to
2.23.8.
- Add annotations for NGINX ingress to configure proxy body size,
read timeout, and send timeout.
These changes enhance the deployment of the application by ensuring
that it uses the latest image version and improves the handling of
larger requests and longer processing time through appropriate NGINX
configuration.
- Enable hostPort for controller
- Disable external controller service
- Add new configuration options for Nginx controller with
allow-snippet-annotations and real-ip handling settings
These changes improve the networking configuration of the Nginx
Ingress controller, enabling better handling of forwarded IP
addresses and ensuring the correct annotations are applied.
It also modifies service accessibility to better fit internal
deployments.
- Change hostname from vkvm-us1 to zgocloud-us1
- Update persistent volume claim name from pg-data-local-vkus1-pvc to pg-data-local-zgus1-pvc
- Modify storage class name to local-zgus1
These updates ensure that the PostgreSQL deployment references the correct
Kubernetes node and persistent volume, aligning with the current
infrastructure and deployment strategy.
- Change storageClass values from 'local-vkus2' to 'local-zgus1'
- Update nodeSelector hostname from 'vkvm-us2' to 'zgocloud-us1'
- Modify persistent volume claim names to reflect new storage class
These changes aim to facilitate a smooth transition to a new hosting environment, ensuring that the resources are correctly allocated and managed.
- Introduce a new StorageClass named `local-data`
- Set provisioner to `rancher.io/local-path`
- Define parameters including `nodePath` and `pathPattern`
- Modify existing StorageClass from `local-data` to `local-zgus1` and include `nodeSelector`
This commit sets up a local storage class to be used within the Kubernetes environment, enabling better management of local volumes.
The new configuration allows for efficient volume binding based on specific node requirements.
- Include tolerations in load.yaml for scheduling
- Allow pods to tolerate specific constraints and improve scheduling
This change enables the containers to schedule on nodes with
restricted taints, enhancing deployment strategies in constrained
Kubernetes environments.
- Remove leading space in the OPENAI_API_MODEL value
- Ensure consistency in YAML formatting
This change corrects the spacing in the OPENAI_API_MODEL
configuration to maintain consistency in the YAML file, which helps
prevent potential issues during processing.
- Change model value from 'o1-mini-2024-09-12' to 'o3-mini'.
This update modifies the model used in the application configuration, which may lead to different behavior or performance improvements depending on the capabilities of the new model.
- Change value of OPENAI_API_MODEL from "o1-mini" to "o1-mini-2024-09-12"
This change updates the model version used in the configuration,
ensuring compatibility with the latest API requirements. The
previous model version will no longer receive updates or support, so
updating this configuration is necessary for continued functionality.
- Create deployment configuration for app3 with 5 replicas.
- Add service to expose app3 on port 3002.
- Define ingress resource for handling external traffic to app3.
- Set up TLS and host configuration for secure traffic.
This commit introduces the necessary Kubernetes manifests for the new
app3 which allows for deployment, service exposure, and ingress
routing, thereby enabling it to be accessed securely over the
specified domain. The deployment includes node affinity rules to
restrict execution environments and enhances the overall service
availability.
- Add tolerations for restricted scheduling
- Set externalTrafficPolicy to 'Local'
These updates enhance the deployment configuration of the nginx-ingress.
They allow better scheduling of pods on nodes with specific taints and
facilitate the preservation of the client source IP for external
traffic.
- Add clhk2.dev.todayaiweather.com to the list of hosts
- Add bwh1.dev.todayaiweather.com to the list of hosts
- Add spus1.dev.todayaiweather.com to the list of hosts
This update allows the application to handle traffic from four
subdomains, providing more flexibility for development and testing.
Each subdomain is routed to the same backend service, ensuring
a consistent experience across these endpoints.
- Modify ingress configuration to redirect requests for /Avatar.jpg
to a specific URL.
- Ensure that the redirection includes query parameters.
- Comment out the old rewrite rule for clarity.
This update addresses the routing for Avatar.jpg, ensuring that
requests are properly directed to the desired image location,
improving accessibility for users.
- Enable snippet annotations for nginx ingress controller
- Comment out a previous configuration snippet that redirects JPG files
This update allows the use of snippet annotations, giving more control
over ingress configurations. The previously commented snippet
suggested a redirect for JPG files but is now ignored to simplify
configuration and focus on the current requirements.
- Change retention period from 5 days to 3 days
This adjustment may help free up storage resources more
quickly, ensuring that older metrics are purged sooner,
leading to better performance and lower storage costs.
