Zend certified PHP/Magento developer

Category products do not load – ‘We can’t find products matching the selection.’ Error keeps coming up at least once a day. Elastic search is running

Magento ver. 2.4.2-p1
Amasty extensions

We used to have Elastic Search crash every once in a while but this is no longer happening since we upgraded to the most powerful server.

We had instances where couple of times a month, when you click on any category – the products would not load and it shows the following Error:

We can’t find products matching the selection.

This now happens once a day.

Usually, running the following fixes the issue:

php bin/magento setup:di:compile 
php bin/magento setup:static-content:deploy -f
php bin/magento cache:clean
php bin/magento cache:flush
php bin/magento indexer:reindex

so I’ve set up cron jobs to run these every 2 hours or so but there is also an issue where running:

php bin/magento setup:di:compile

does not work (it gives an Error about some generated folder not being empty). When that happens, running it in this order seems to work:

php bin/magento cache:clean
php bin/magento cache:flush
php bin/magento indexer:reindex
php bin/magento setup:static-content:deploy -f
php bin/magento setup:di:compile 
php bin/magento setup:static-content:deploy -f

But this still leaves us with website downtime. The products on the front page load always but the issue appears for all products in all categories when it comes up.

I’ve googled this and tried running the following:

select * from mget_catalog_eav_attribute cea join mget_eav_attribute ea on ea.attribute_id = cea.attribute_id and backend_type in ('text', 'varchar') where cea.is_filterable = 1;

MySQL returned an empty result set (i.e. zero rows). (Query took 0.0007 seconds.)

update mget_catalog_eav_attribute cea 
join mget_eav_attribute ea
  on ea.attribute_id = cea.attribute_id 
  and backend_type in ('text', 'varchar')
set cea.is_filterable = 0
where 
  cea.is_filterable = 1

0 rows affected. (Query took 0.0008 seconds.)

How can I narrow down where the problem is coming from and how to fix it permanently?