Magento Full Page Cache Cronjob
With full page cache auto generation turned on, we found the cron job is taking too long to run on a large site and is holding up all other cron jobs, so we have removed this cron job (enterprise_page_cache_crawler) from Magento cron and have created a separate cron job specifically for the cache crawler. The [...]
Magento Full Page Caching doesn’t cache homepage
We recently had an issue where our homepage was allot slower than the rest of the site. We had full page caching turned on for everything but it didn’t work for the homepage? It turns out that Magento full page cache doesn’t actually cache all the pages just specifically catalog and product pages??? To get around [...]
Magento putting a hole through full page cache
http://stackoverflow.com/questions/8126548/trying-get-dynamic-content-hole-punched-through-magentos-full-page-cache http://tweetorials.tumblr.com/post/10160075026/ee-full-page-cache-hole-punching http://magentophp.blogspot.co.uk/2011/02/magento-enterprise-full-page-caching.html http://oggettoweb.com/blog/customizations-compatible-magento-full-page-cache/
Speeding up Magento
This page will list everything we have found on speeding up your copy of Magento. Before you can start to speed up Magento you need to know how fast it is now, here are a few options for testing speed. http://newrelic.com/ Get the page speed: http://tools.pingdom.com/fpt/ http://gtmetrix.com/ Speed Options Memcache http://blog.nexcess.net/2012/02/24/magento-enterprise-and-memcached/ Cache http://www.fabrizio-branca.de/magento-zend-frameworks-twolevels-cache-backend-mess.html http://magebase.com/magento-tutorials/improving-the-file-cache-backend/ Varnish [...]
MySQL Get the Database file sizes
Get the size in mb’s of the tables in your database from a SQL Query This is a very useful query to get the size of all the tables in a database: select TABLE_NAME, sum( data_length + index_length ) / 1024 / 1024 "Data Base Size in MB" from information_schema.TABLES where TABLE_SCHEMA = "database_name" group [...]
Magento checking object initiated
Below is a bit of code to check whether the instance of product is loaded and if not load it. Nice tidy way of making sure your instance is initiated, /** * Retrieve product view page url * * @param mixed $product * @return string */ public function getProductUrl($product) { if ($product instanceof Mage_Catalog_Model_Product) { [...]
Magento just one url for a product
*** This is incomplete at the moment as we are still doing it, I will update this page as I go. Only do this if you know what your doing, no liability *** By default Magento allows multiple url’s for a product, this is actually a problem for many sites as google see’s this as duplicate content on the [...]
Magento Get the root categories for all the products
Get all the root categories into a temporary table DROP TEMPORARY TABLE IF EXISTS prodRootCat; CREATE TEMPORARY TABLE IF NOT EXISTS prodRootCat ( rootname varchar(50) DEFAULT NULL ,KEY(product_id) ,KEY(sku) ,KEY(rootid) ) SELECT DISTINCT catalog_category_product.product_id ,catalog_product_entity.sku AS sku ,SUBSTRING_INDEX(SUBSTRING_INDEX(catalog_category_entity.path, '/', 3), '/', -1) AS rootid , '' AS rootname FROM catalog_product_entity JOIN catalog_category_product ON catalog_product_entity.entity_id = [...]
Favorite Linux Commands
This is a list of some of my favorite Linux commands: Get a count by file extension: find ./ -type f | grep -E ".*\.[a-zA-Z0-9]*$" | sed -e 's/.*\(\.[a-zA-Z0-9]*\)$/\1/' | sort | uniq -c | sort -n
Magento dealing with Illegal Characters
We have found a few dodgy characters after importing lots of products into Magento. There are several ways to resolve these characters. This sql will convert all the characters to Binary and then to latin1. UPDATE catalog_product_entity_varchar SET `value` = CONVERT(CONVERT(`value` USING latin1) USING BINARY) ; Additional Information SHOW VARIABLES LIKE "%character_set%";
Magento URL Rewrite
The table core_url_rewrite is very useful as it stores all the Apache rewrites for your Magento website. If you select data from the core_url_rewrite table you will see all the url rewrites. select * from core_url_rewrite; In the options there is a flag RP, this stands for Rewrite Permanent. RP = Rewrite Permanent These rewrite [...]
Magento change product status
If you have lots of products you need to disable then this can be done very easily via the following command line scripts: First you need to get the main product attribute class which I have written. Then just add a function to do what ever you want to the product attributes e.g. try { // Disable [...]
Magento Update product attributes script
This is one of my favorite Magento scripts and I use if everywhere to update Magento product attributes quickly at the command line. I have updated the script so you can update any type of attribute (Except static). **** Make sure you know what your doing before using scripts like this, there is no liability if you using this [...]
Category Indexing
It looks like the main table which holds the indexes for the category is: catalog_category_product_index There will be a entry for every products category and the products hierarchical categories, so if I have a product in the following category structure: Pets / Dog / Dog Food There will be an entry for each category in the [...]
Magento check a table exists via isTableExists
It took me a while to get this working correctly so hopefully this will save somebody some time, when using magento’s isTableExists you need to make sure you haven’t quoted the table name and if you are using a schema pass it separately to the method e.g. $table="`customer_entity`"; $schema="`database`"; Mage::getSingleton('core/resource') ->getConnection('core_write') ->isTableExists(trim($table,'`'),trim($schema,'`')));
Useful Methods/Functions
/** * Get all the products attribute names */ public function getProductAttributes() { $collection = Mage::getResourceModel('catalog/product_attribute_collection') ->toArray(); foreach($collection as $c) { foreach($c as $v) $attName[] = $v['attribute_code']; } //foreach print_r($attName,true); return; } /** * Get the downloadable products number of pages */ public function getNumProdPages($iItemsPerPage=10) { return Mage::getModel('catalog/product')->getCollection() ->addAttributeToFilter('type_id', 'downloadable') ->addAttributeToSelect('entity_id') ->setPageSize($iItemsPerPage) ->getLastPageNumber(); [...]
Debugging Magento
One of the most important things you need to do to develop in Magento is learn to debug, so we are going to put all our tips and tricks in here. When you run a call to a collection you can view the select statement by using getSelect e.g. $collection = Mage::getModel('review/review') ->getCollection() ->addAttributeToSelect('status_id', '2'); [...]
Cron Jobs
Virtually everything you need to know about cron jobs in Magento http://www.magentocommerce.com/wiki/1_-_installation_and_configuration/how_to_setup_a_cron_job Viewing all magento cronjobs <?php include 'app/Mage.php'; Mage::app(); $crontab = Mage::getConfig()->getNode('crontab/jobs'); print_r($crontab);
All products in a category by category id
All products in a category by category id /* All products in a category */ SELECT p.entity_id AS prod, prod_varchar.value, p.sku AS sku, cp.category_id AS cp_category_id, cp.`position` AS cp_position, c.`entity_id` AS c_category_id, cat_varchar.value AS CatName, c.`entity_type_id` AS c_entity_type_id, c.`attribute_set_id` AS c_attribute_set_id, c.`parent_id` AS c_parent_id, c.`path` AS c_path, c.`position` AS c_position, c.`level` AS c_level, c.`children_count` AS [...]
Return a products category details
Return a products category details /* Search for a products category */ SELECT p.entity_id AS prod, prod_varchar.value, p.sku AS sku, cp.category_id AS cp_category_id, cp.`position` AS cp_position, c.`entity_id` AS c_category_id, cat_varchar.value AS CatName, c.`entity_type_id` AS c_entity_type_id, c.`attribute_set_id` AS c_attribute_set_id, c.`parent_id` AS c_parent_id, c.`path` AS c_path, c.`position` AS c_position, c.`level` AS c_level, c.`children_count` AS c_children_count FROM catalog_product_entity [...]
Get number of products in each category
Get number of products in each category /* Show products in each categories */ SELECT c.`entity_id` AS CatId, cat_varchar.value AS CatName, count(cp.product_id) as NumProducts FROM catalog_category_entity AS c LEFT JOIN catalog_category_product AS cp ON c.entity_id = cp.category_id LEFT JOIN catalog_category_entity_varchar AS cat_varchar ON c.entity_id = cat_varchar.entity_id AND cat_varchar.attribute_id = 111 WHERE 1 #and c.entity_id = [...]
Cool MySQL Statements
SQL Script’s which I think are great Dump certain tables which start with a prefix/name just replace all the text in UPPERCASE with your specific values: mysqldump -u USER -p –database DATABASE –tables $(mysql -u USER -p -D DATABASE -Bse "show tables like 'PREFIX%'") > dump_file.sql Find a column in a database (Love this [...]
Adding Flowplayer video
How to add Flowplayer video to Magento The Wysiwg editor in Magento changes code and so you cannot add the flowplayer code in this way. What we need to do is add a phtml page and then put a link to it in the cms page. Add flowplayer to site Go to http://flowplayer.org/download/index.html [...]
Solr Cheatsheet
Useful Solr Stuff To view all the field that are indexed go to the admin console and put luke at the end of the url e.g. http://localhost:8983/solr/core/admin/luke To do a simple search via the admin console: fieldname:"search_string" This will return the results for the search term search_string fro the field fieldname.
Magento One Page Checkout
Magento One Page Checkout References Removing Checkout First Page http://www.magentocommerce.com/magento-connect/skip-checkout-step-1.html Removing Shipping Address http://www.bluehorse.in/amit-bera/bluehorse-stuffs/removed-shipping-address-and-shipping-method-from-onepage-checkout-in-magento/ Removing Billing Address http://www.webspeaks.in/2010/12/bypassing-billing-informationfirst-step.html Remove Shipping Method http://www.excellencemagentoblog.com/magento-onestep-checkout-remove-shipping-method-step Adding a New Step http://www.excellencemagentoblog.com/magento-onestep-checkout-add-step http://www.unexpectedit.com/magento/add-new-customer-attribute-onepage-magento-checkout http://inchoo.net/ecommerce/magento/adding-a-new-tab-under-one-page-checkout-full-working-module/ Creating new Fields http://www.blog.magepsycho.com/magento-utility-function-how-to-easily-create-select-box-for-drop-down-attributes/ Creating a Username Field http://www.magentocommerce.com/magento-connect/username-support-login-register-checkout-by-diglin.html Other Stuff AJAX add product to basket http://www.excellencemagentoblog.com/magento-add-product-to-cart-ajax
Error when turning stock off for large products base
When we try and turn off “Manage Stock” for a large product base we are getting the error message “Lock wait timeout exceeded”, screen shot below. What we tried to fix the problem We checked the PHP setting with a another development environment we have and they are the same and that development environment is not timing out, [...]
Indexing from command line
Re-indexing your Magento data can make a bit difference to the speed of your site so it is very important to keep your indexes up to date. This can be done easily on a small site from the admin console but for larger sites with many thousands of products and customers it is best to perform the indexes from a SSH prompt. Here [...]
Generally Useful Stuff
Fixes common issues: http://www.siteground.com/tutorials/magento/magento_issues.htm
Calls to MySQL in Magento
Get all Methods Function to get all the available direct database call methods, there is a search option too, so you would call it like: getClassmethods("insert"); function getClassmethods($command="") { if(!empty($command)) { $results=array(); $methods=get_class_methods($this->_getConn()); foreach ($methods as $v){ if(stripos($v, $command) !== false){ $results[]=$v; } //if } //foreach echo '<pre>'; print_r($results); echo '</pre>'; } else { echo [...]
Magento & Solr
Solr is a very powerful search engine which can be used with Magento Enterprise, it is a stand-a-lone search engine which usually runs on a separate server to Magento. It can be used instead of the usual MySQL search to find products and categories allot faster. We implemented Solr with Magento Enterprise and have had [...]
Tutorials & References
This is a list of tutorials & References I have found along the way on Magento development, some are good and some not so good, I will try and comment to say what I think of them, this is of course a personal opinion. Tutorials http://www.pierrefay.com/magento-create-block-44 This seems like a very comprehensive tutorial on how to create [...]
Get all Categories with full path from Database
This will return all the categories with their paths down to the 9th tier, if you have deeper categories just duplicate the 1 to 9 bits… SET @AttribIdForName := 111; # Name of the category SET @StoreId := 0; DROP TEMPORARY TABLE IF EXISTS mag_hier_cats; CREATE TEMPORARY TABLE mag_hier_cats (Level_Name_2 varchar(50),Level_Name_3 varchar(50),Level_Name_4 varchar(50),Level_Name_5 varchar(50),Level_Name_6 varchar(50),Level_Name_7 [...]
Whats in the config.xml file?
So whats in the config.xml file? A very good question, we are assuming you know what it is and why it’s used here I am trying to get some resources together to explain what goes in it. http://www.magentocommerce.com/wiki/5_-_modules_and_development/reference/module_config.xml The config.xml files basically allows us to find the modules etc that we have created, [...]
Eclipse IDE
Well we have decided to use the IDE product Eclipse for our Magento development after a few issues with NetBeans just hanging whilst trying to refresh the work space. I must admit Eclipse is not very intuitive for me but I will see how I get on. Plugins This is a list of the plugins [...]
An error occurred while saving this configuration: SQLSTATE[HY000]: General error: 1114 The table ‘catalogsearch_result’ is full
I have been getting this error message when I try and change the catalog search settings in the admin console: An error occurred while saving this configuration: SQLSTATE[HY000]: General error: 1114 The table ‘catalogsearch_result’ is full The fix for this is to update your MySQL settings so that the autoextend maximum is not reached. The [...]
MD5, Hash and Crypt
Today I will be trying to change the way Magento saves customer passwords from Hash to Crypt, we need to do this as we have legacy data which has been using a Perl module to Crypt the passwords and we need to import the customer into the new Magento website and allow them to login using their [...]
SQL Injections
SQL Injections A major problem on websites is people who have nothing beter to do than distroy, mainly because they cannot create themselves. We here are a few resourcesto hopefully help you slow them down and maybe stop them. https://www.owasp.org/index.php/SQL_Injection_Prevention_Cheat_Sheet
Magento Adding new customer attributes
It is quite tricky to add attributes to customers and it can take quite a while to find out how to go about this as there are several methods of achieving this. Adding directly into the eav database Adding directly to database via a mysql_upgrade file Adding via php code From our opinion the best [...]