429 Too Many Requests

Resolved

(@swapot)

I am importing variable products via the built-in Woocommerce product import. Each product contains between 170-250 variables, and the CSV files are around 130 kb each.

On my previous (shared) web hosting, these would each take a minute or two to upload. I changed hosting to VPS with high frequency server: 2 vCPUs/4 GB Memory/3 TB Bandwidth/128 GB Storage. From what I researched, I thought this would have room to grow for one woocommerce website.

However, I am having high loads when managing the WP media library and when I import a product from CSV, it is now taking 15 minutes or more. Since I have 500 more products to import, I am looking for a solution.

It is not my home internet – I have upload speeds of 8-10 Mbps. The new server is optimized, for wordpress/woocommerce with caching, etc. I have also tried uploading with most all plugins deactivated and with caching and CDN turned off but it made no difference.

Are two vCPUs not enough for a Woocommerce site that has 800 products, all of which are variable products with 170-250 variations each? Or is there something slowing down WordPress/Woocommerce/Database when I’m importing a product? I’m not sure how to troubleshoot something like this.

Any help appreciated.

System Status Report

### WordPress Environment ###

WordPress address (URL): [Redacted]
Site address (URL): [Redacted]
WC Version: 9.5.1
Legacy REST API Package Version: ✔ 1.0.4
Action Scheduler Version: ✔ 3.8.2
Log Directory Writable: ✔
WP Version: 6.7.1
WP Multisite: –
WP Memory Limit: 1 GB
WP Debug Mode: ✔
WP Cron: ✔
Language: en_US
External object cache: ✔

### Server Environment ###

Server Info: LiteSpeed
PHP Version: 8.3.15
PHP Post Max Size: 256 MB
PHP Time Limit: 300
PHP Max Input Vars: 1000
cURL Version: 8.5.0
OpenSSL/3.0.13

SUHOSIN Installed: –
MySQL Version: 11.4.4-MariaDB-ubu2404
Max Upload Size: 256 MB
Default Timezone is UTC: ✔
fsockopen/cURL: ✔
SoapClient: ✔
DOMDocument: ✔
GZip: ✔
Multibyte String: ✔
Remote Post: ✔
Remote Get: ✔

### Database ###

[REDACTED]

### Post Type Counts ###

acf-field: 4
acf-field-group: 1
athemes_hf: 2
attachment: 178954
custom_css: 1
mailpoet_page: 1
ml-slide: 4
ml-slider: 1
nav_menu_item: 2
page: 10
post: 2
product: 343
product_variation: 66906
shop_order: 27
wp_font_face: 33
wp_font_family: 12
wp_global_styles: 1
wp_navigation: 1
wp_template: 2
wpforms: 1
yith_wcan_preset: 2

### Security ###

Secure connection (HTTPS): ✔
Hide errors from visitors: ✔

### Active Plugins (13) ###

FiboSearch - AJAX Search for WooCommerce (Pro): by FiboSearch Team – 1.29.0
All in One SEO Pro: by All in One SEO Team – 4.7.7
aThemes Blocks: by aThemes – 1.0.13
Botiga Pro: by aThemes – 1.4.10
LiteSpeed Cache: by LiteSpeed Technologies – 6.5.4
Smart Slider 3 Pro: by Nextend – 3.5.1.26
Redis Object Cache: by Till Krüss – 2.5.4
WooCommerce.com Update Manager: by Automattic – 1.0.3
Amazon S3 Storage for WooCommerce: by Kestrel – 3.0.3
WooCommerce Stripe Gateway: by Stripe – 9.0.0
WooCommerce Legacy REST API: by WooCommerce – 1.0.4
WooCommerce PayPal Payments: by WooCommerce – 2.9.5
WooCommerce: by Automattic – 9.5.1

### Inactive Plugins (10) ###

Add From Server: by Dion Hulse – 3.4.5
Auto Image Attributes From Filename With Bulk Updater: by Arun Basil Lal – 4.4
BEAR – Bulk Editor and Products Manager Professional for WooCommerce: by realmag777 – 1.1.4.5
Complianz | GDPR/CCPA Cookie Consent: by Really Simple Plugins – 7.1.5
Hide My Site: by Justin Saad – 2.2
MailPoet: by MailPoet – 5.5.2
Merchant Pro: by aThemes – 1.11.1
Plugin Organizer: by Jeff Sterup – 10.1.10
WPForms Lite: by WPForms – 1.9.2.3
WP Rocket: by WP Media – 3.17.3

### Dropin Plugins () ###

object-cache.php: Redis Object Cache Drop-In

### Settings ###

Legacy API Enabled: ✔
Force SSL: –
Currency: USD ($)
Currency Position: left
Thousand Separator: ,
Decimal Separator: .
Number of Decimals: 2
Taxonomies: Product Types: external (external)
grouped (grouped)
simple (simple)
variable (variable)

Taxonomies: Product Visibility: exclude-from-catalog (exclude-from-catalog)
exclude-from-search (exclude-from-search)
featured (featured)
outofstock (outofstock)
rated-1 (rated-1)
rated-2 (rated-2)
rated-3 (rated-3)
rated-4 (rated-4)
rated-5 (rated-5)

Connected to WooCommerce.com: ✔
Enforce Approved Product Download Directories: –
HPOS feature enabled: –
Order datastore: WC_Order_Data_Store_CPT
HPOS data sync enabled: –

### Logging ###

Enabled: ✔
Handler: Automattic\WooCommerce\Internal\Admin\Logging\LogHandlerFileV2
Retention period: 30 days
Level threshold: –
Log directory size: 644 KB

### WC Pages ###

Shop base: #6 - /shop/
Cart: #7 - /cart/ -  Contains the woocommerce/cart block
Checkout: #8 - /checkout/ -  Contains the woocommerce/checkout block
My account: #9 - /my-account/
Terms and conditions: #3612 - /terms-and-conditions/

### Theme ###

Name: Botiga Child
Version: 1.0
Author URL: http://athemes.com
Child Theme: ✔
Parent Theme Name: Botiga
Parent Theme Version: 2.2.15
Parent Theme Author URL: https://athemes.com
Theme type: Classic theme
WooCommerce Support: ✔

### Templates ###

Overrides: botiga-child/woocommerce/emails/customer-completed-order.php
botiga-child/woocommerce/emails/customer-invoice.php
botiga-child/woocommerce/emails/email-header.php


### WooCommerce PayPal Payments ###

Onboarded: ✔
Shop country code: US
WooCommerce currency supported: ✔
Advanced Card Processing available in country: ✔
Pay Later messaging available in country: ✔
Webhook status: –
PayPal Vault enabled: –
ACDC Vault enabled: –
Logging enabled: –
Reference Transactions: –
Used PayPal Checkout plugin: –
Subscriptions Mode: Disabled
PayPal Shipping Callback: –
Apple Pay: –
Google Pay: –
Fastlane: –

### Admin ###

Enabled Features: activity-panels
analytics
product-block-editor
coupons
core-profiler
customize-store
customer-effort-score-tracks
import-products-task
experimental-fashion-sample-products
shipping-smart-defaults
shipping-setting-tour
homescreen
marketing
mobile-app-banner
onboarding
onboarding-tasks
pattern-toolkit-full-composability
product-custom-fields
remote-inbox-notifications
remote-free-extensions
payment-gateway-suggestions
printful
shipping-label-banner
subscriptions
store-alerts
transient-notices
woo-mobile-welcome
wc-pay-promotion
wc-pay-welcome-page
launch-your-store

Disabled Features: product-data-views
experimental-blocks
coming-soon-newsletter-template
minified-js
product-pre-publish-modal
settings
async-product-editor-category-field
product-editor-template-system
blueprint
reactify-classic-payments-settings
use-wp-horizon
add-to-cart-with-options-stepper-layout

Daily Cron: ✔ Next scheduled: 2024-12-31 18:53:37 -10:00
Options: ✔
Notes: 59
Onboarding: completed

### Action Scheduler ###

Complete: 16,842
Oldest: 2024-11-29 09:37:34 -1000
Newest: 2024-12-30 08:57:33 -1000

Failed: 1,829
Oldest: 2023-08-12 11:12:02 -1000
Newest: 2024-12-18 09:13:52 -1000

Pending: 6
Oldest: 2024-12-30 08:58:03 -1000
Newest: 2025-01-04 03:31:46 -1000


### Status report information ###

Generated at: 2024-12-30 08:57:58 -10:00

The page I need help with: [log in to see the link]

Viewing 10 replies - 1 through 10 (of 10 total)

Plugin Support Mahfuzur Rahman(woo-hc)
(@mahfuzurwp)

4 months, 1 week ago

@swapot,

Thanks for reaching out! I understand how frustrating it can be when imports take longer than expected. Let me provide some insights and recommendations to help improve the import process.

Key Factors to Consider:
1. Database Performance: Importing variable products with 170-250 variations each can create a heavy load on the database since each variation is treated as a separate entry. Optimizing your database tables (e.g., wp_posts, wp_postmeta) could help reduce the strain during imports.

2. Server Resources: While your VPS specs (2 vCPUs and 4 GB RAM) are solid for most WooCommerce sites, bulk operations like imports can require additional resources. Upgrading to 4+ vCPUs could help if you’re regularly importing large datasets.

3. PHP Configuration: Ensure your server’s PHP limits are set high enough:
• max_execution_time → 300 or higher
• memory_limit → 512M or higher
• upload_max_filesize → At least the size of your CSV file

Recommendations:
• Break Down the File: Try splitting your CSV into smaller files (e.g., 50-100 products per file) to reduce the load on your server during imports.
• Database Optimization: Make sure your database is optimized. Your hosting provider may assist with this.
• Monitor Server Load: Use a tool like New Relic or server logs to pinpoint bottlenecks (e.g., high CPU usage or slow database queries) during the import process.

If the issue persists, you might also want to contact your hosting provider to review the server’s performance during imports.
Thread Starter swapot
(@swapot)

4 months ago
@mahfuzurwp

Thank you for the quick response. I really appreciate your things to consider and suggestions. I few comments/questions to further clarify:

Break Down the File: Try splitting your CSV into smaller files (e.g., 50-100 products per file) to reduce the load on your server during imports.

Each CSV is only one product per file. That is why I was surprised to have it take 15 minutes to import one product (what before took a minute or two).
1. PHP Configuration – Here’s what I have:
- max_execution_time → 300
- memory_limit → 1024
- upload_max_filesize → 256 MB (my CSV files are about 130kb each)
Database Optimization: Make sure your database is optimized. Your hosting provider may assist with this.

I will check with my hosting provider, but can a database optimization plugin help or possibly hurt?
Plugin Support shahzeen(woo-hc)
(@shahzeenfarooq)

4 months ago

Hi there!

I understand your concern, and thank you for sharing information about your PHP configuration. For testing purposes, could you please share the CSV file you are trying to upload? This will allow me to attempt the upload on my test site and replicate the issue. This way, I can determine whether the issue lies with the hosting settings or if there is something else on your site causing the problem.

Additionally, a database optimization plugin can be helpful in some cases, as it can clean up unnecessary data like post revisions, trashed items, and transient entries. However, always create a full backup of your database before running any optimization. This ensures you can restore your site if something goes wrong.

Consider using a dedicated import tool like WP All Import or Import Export Suite for WooCommerce.

Let me know if you still face the same issue when importing products using a plugin.

Thread Starter swapot
(@swapot)

4 months ago

Is there a way to share the CSV file not publicly in this forum?

Plugin Support Zubair Zahid (woo-hc)
(@doublezed2)

4 months ago

Hello swapot,

Thank you for your reply.

Please note that this is a public forum and all information shared is available publicly.
There is no private way to share information with us.

<span style=”box-sizing: border-box; margin: 0px; padding: 0px;”>If you like, you may use the https://quickforget.com tool, which allows you to remove shared infor</span>mation automatically after a certain time.

Let me know if you have more questions.

Best regards.

Thread Starter swapot
(@swapot)

3 months, 4 weeks ago

Hi Woo team,

I have included a link to download one of my variable product CSV files. It will only be available for download for the next 72 hours. You won’t have the 252 webp images that this CSV file will look for to create the variable product, but I think it will work since the links are full links to the URL.

https://quickforget.com/s/07eff6461563977f9ebf455e358f85f8c1d905488c4e56eb

I am interested to know how long (in minutes/seconds) it takes you to create this product via the standard Woocommerce “import” button in your environment.

I am trying to determine why these are now taking so long to import (one variable product at a time), when previously, on different server hosting (that is reportedly slower), it was much faster.

Thank you.

Thread Starter swapot
(@swapot)

3 months, 4 weeks ago

A successful import for me starts with the following:

“Import complete! 1 product imported. 252 variations imported…”

Plugin Support Zubair Zahid (woo-hc)
(@doublezed2)

3 months, 4 weeks ago

Hello swapot,

Thank you for your reply.

I imported the sample CSV file and started a stopwatch.
The file took 4 minutes and 17 seconds to import.

May I ask how much time does this file take on your site?

I suggest you consider a solution where you have fewer variations for each product.
That will help speed up the import process.

Feel free to contact us again if you have any questions.

Best regards.
Thread Starter swapot
(@swapot)

3 months, 4 weeks ago
Hi Zubair,

It used to take me 2-4 minutes to upload these.

Since I changed hosting to VPS with high frequency server: 2 vCPUs/4 GB Memory/3 TB Bandwidth/128 GB Storage, it was taking 15 minutes per file.

I just recently moved a copy of the site to a shared hosting environment with access to 16 vCPUs, unlimited PHP workers, etc. and they are taking about 8 minutes to import.

What I am trying to determine is:
1. Is the processing speed of the server/host mostly to blame for the slower import speeds? -or-
2. Do I have some bottlenecks in my database that I need to hire a programmer to clean up and optimize? (I have ran WordPress’ database optimize tool and tried two different database optimization plugins, but those have not made a difference). -or-
3. Something else
I realize that 252 variations is a LOT of variations for one product, but that is what they are, so I cannot adjust that.

Open to suggestions.
Plugin Support Mahfuzur Rahman(woo-hc)
(@mahfuzurwp)

3 months, 4 weeks ago
Hi @swapot ,

Thanks for sharing the detailed information about your setup and the issue you’re encountering. Based on your description, here are some possible factors contributing to the slower import speed:

1. Server Resources:
Your VPS with 2 vCPUs and 4 GB memory might not be sufficient for handling imports of variable products with 170–250 variations. Each variation requires multiple database queries and updates, which can be resource-intensive.

2. Database Performance:
The database may have accumulated overhead or inefficiencies that are slowing down operations during imports. Despite running optimization tools, a manual investigation into database bottlenecks might be necessary.

4. File Processing Overhead:
If the import involves uploading images or other media, server performance (like disk I/O and PHP limits) could also impact the speed.

To address these issues, we recommend the following steps:
- Try a third-party import plugin like WP All Import or similar.
- Consider a database expert: Optimizing your database manually can make a noticeable difference. Based on feedback from users, we recommend reaching out to one of the services on our Customization page for expert assistance.
- Test with increased server resources: If possible, temporarily increase the VPS resources (e.g., 4 vCPUs, 8 GB memory) to see if it impacts the import speed.