I am trying to integrate RabbitMQ with Magento 2 to access bulk endpoints
Bitnami Magento ver. 2.4.6-p2
I have successfully set up RabbitMQ with my Magento 2 instance, and now bulk requests are being sent to the queue. However, I’m facing challenges with requests processing – either the requests aren’t processed as expected, or there’s an error that I’m not able to pinpoint.
My magento-values.yaml file:
magentoUsername: user
magentoPassword: p@ssword123
magentoHost: magento.company.com
mariadb:
auth:
rootPassword: p@ssword123
password: p@ssword123
ingress:
enabled: true
pathType: Prefix
hostname: magento.company.com
path: /
annotations:
kubernetes.io/ingress.class: azure/application-gateway
appgw.ingress.kubernetes.io/appgw-ssl-certificate: dev-certs-com
appgw.ingress.kubernetes.io/use-private-ip: "true"
tls: true
extraVolumes:
- name: cron-file
configMap:
name: magento-cron
extraVolumeMounts:
- name: cron-file
mountPath: /etc/cron.d
readOnly: true
My rabbitmq-deployment.yaml file:
apiVersion: apps/v1
kind: Deployment
metadata:
name: rabbitmq
namespace: dev-magento-ns
spec:
replicas: 1
selector:
matchLabels:
app: rabbitmq
template:
metadata:
labels:
app: rabbitmq
spec:
containers:
- name: rabbitmq
image: rabbitmq:3-management
ports:
- containerPort: 5672
- containerPort: 15672
---
apiVersion: v1
kind: Service
metadata:
name: rabbitmq
namespace: dev-magento-ns
spec:
selector:
app: rabbitmq
ports:
- name: amqp
protocol: TCP
port: 5672
targetPort: 5672
- name: management
protocol: TCP
port: 15672
targetPort: 15672
type: LoadBalancer
After this I ran this:
php bin/magento setup:config:set --amqp-host="52.234.243.215" --amqp-port="5672" --amqp-user="guest" --amqp-password="guest" --amqp-virtualhost="/"
Then I ran this: php bin/magento queue:consumers:start async.operations.all
And when I check it in magento2 pod, I get:
I have no name!@my-magento-59f6bdcbd5-pp8vp:/bitnami/magento$ ps aux | grep 'queue:consumers:start'
1001 589 0.0 0.9 689768 162860 pts/0 S Dec11 0:02 php bin/magento queue:consumers:start async.operations.all
1001 288196 0.0 0.0 3240 1664 pts/16 S+ 10:52 0:00 grep queue:consumers:start
I have no name!@my-magento-59f6bdcbd5-pp8vp:/bitnami/magento$
My env.php:
.
.
.
'queue' => [
'consumers_wait_for_messages' => 1,
'amqp' => [
'host' => '53.243.234.211',
'port' => '5672',
'user' => 'guest',
'password' => 'guest',
'virtualhost' => '/'
]
],
.
.
.
'cron_consumers_runner' => [
'cron_run' => true,
'max_messages' => 10000,
'consumers' => [
'product_action_attribute.update',
'product_action_attribute.website.update',
'exportProcessor',
'media.storage.catalog.image.resize',
'matchCustomerSegmentProcessor',
'codegeneratorProcessor',
'sales.rule.update.coupon.usage',
'sales.rule.quote.trigger.recollect',
'product_alert',
'staging.synchronize_entity_period',
'negotiableQuotePriceUpdate',
'sharedCatalogUpdatePrice',
'sharedCatalogUpdateCategoryPermissions',
'inventory.source.items.cleanup',
'inventory.mass.update',
'inventory.reservations.cleanup',
'inventory.reservations.update',
'inventory.reservations.updateSalabilityStatus',
'inventory.indexer.sourceItem',
'inventory.indexer.stock',
'media.content.synchronization',
'media.gallery.renditions.update',
'media.gallery.synchronization',
'placeOrderProcessor',
'purchaseorder.toorder',
'purchaseorder.transactional.email',
'purchaseorder.validation',
'quoteItemCleaner',
'inventoryQtyCounter',
'async.operations.all'
]
]
];
Then I performed a bulk operation using curl:
# Get bearer token
C:Usersikrkhanschemamagento>curl -X POST "https://magento.company.com/rest/V1/integration/admin/token" -H "Content-Type:application/json" -d "{"username":"user", "password":"p@ssword123"}"
"adsadsdadadadadsadsadLCJleHAiOjE3MDI1NjAzNjV9.5D3zk5fyTUHrknTLcPEVlYlgl7fNPSbwWk7H-nqvbZ4"
# POST products in bulk
C:Usersikrkhanschemamagento>curl -X POST "https://magento.company.com/rest/all/async/bulk/V1/products" -H "Authorization: Bearer adsadsdadadadadsadsadLCJleHAiOjE3MDI1NjAzNjV9.5D3zk5fyTUHrknTLcPEVlYlgl7fNPSbwWk7H-nqvbZ4" -H "Content-Type: application/json" -d @bulk_add_test.json
{"bulk_uuid":"13f2a323-cfbe-47c5-a54d-be6d8dc141bb","request_items":[{"id":0,"data_hash":"684841f654k2jbk2j4bjk2b42k42jb8e9f75684d3","status":"accepted"},{"id":1,"data_hash":"b823a2adsa4342kj24kjn422kj42n2kj32370019c4c85e9","status":"accepted"},{"id":2,"data_hash":"619bcd1536e501b4590877a0909750bcfecd2912882ae2e92a38392b25ea6bc8","status":"accepted"},{"id":3,"data_hash":"5dadawad016106bc9c59d1b566f16awdadadawdadaw34a4","status":"accepted"},{"id":4,"data_hash":"eb7bbc8b26542bkj2b4kj2b4k24bkj2b42k4j42jndee4c4af2e6","status":"accepted"},{"id":5,"data_hash":"bc3742050682f6464d3be2cad56b277996f9b057e1ea0ddd4ef01f70686fb3ad","status":"accepted"},{"id":6,"data_hash":"bd480a8a517ac76j2kbk42bkj4b2kjbkj2442j5d732adc","status":"accepted"},{"id":7,"data_hash":"ff54d258a701904b2kjjk24b2kj4bj2b4jk244d337cd722f","status":"accepted"},{"id":8,"data_hash":"e0a65a6764aa53a6b5kj2b4kj2bkj42bjk4b2521c126832371006","status":"accepted"}],"errors":false}
C:Usersikrkhanschemamagento>curl -X GET "https://magento.company.com/rest/V1/bulk/13f2a323-cfbe-47c5-a54d-be6d8dc141bb/status" -H "Authorization: Bearer adsadsdadadadadsadsadLCJleHAiOjE3MDI1NjAzNjV9.5D3zk5fyTUHrknTLcPEVlYlgl7fNPSbwWk7H-nqvbZ4"
{"operations_list":[{"id":0,"status":4,"result_message":null,"error_code":null},{"id":1,"status":4,"result_message":null,"error_code":null},{"id":2,"status":4,"result_message":null,"error_code":null},{"id":3,"status":4,"result_message":null,"error_code":null},{"id":4,"status":4,"result_message":null,"error_code":null},{"id":5,"status":4,"result_message":null,"error_code":null},{"id":6,"status":4,"result_message":null,"error_code":null},{"id":7,"status":4,"result_message":null,"error_code":null},{"id":8,"status":4,"result_message":null,"error_code":null}],"user_type":2,"bulk_id":"13f2a323-cfbe-47c5-a54d-be6d8dc141bb","description":"Topic async.magento.catalog.api.productrepositoryinterface.save.post","start_time":"2023-12-14 12:33:15","user_id":1,"operation_count":9}
But now it is stuck in pending status:
What steps can I take to diagnose and resolve this issue?
Any pointers are welcomed. Thanks!
