Many times, there is a knock on Symfony framework that is slow. Frequently you will hear quips like “Symfony has a great set of features for building enterprise applications but performs worse than <insert your framework here>”. There are also many benchmarks that will take a trivial use case and an untuned Symfony and paint a sorry picture.
But we have rolled out pretty zippy applications that handle tons of loads using Symfony. We will share a few tips and tricks that we have used in our apps.
First off, performance is an important aspect and cannot be an afterthought. Often teams will finish the functionality and then do a 'performance testing and fixing' phase. It goes without saying that it is the wrong approach. The overall performance of the application is a culmination of many design decisions and implementation tricks and hence performance should be an exercise throughout the life cycle of the project.
Given that at Ideas2IT, we offer performance and scalability as a service offering itself and invested in a partnership with AppDynamics, we have developed a framework that follows the cycle of Estimate, Measure, Profile, Implement/Improve – Repeat, throughout the life cycle of a project to achieve desirable performance and scalability.
Estimate:It is important to get an estimate of what is expected of the system in terms of
- Load
- Data Volume
- Availability
- Cost constraints
If you don't do this exercise, we will under-tune the system. And sometimes might over plan which can be a costly exercise.
Measure:Never attempt to fix performance without measuring the performance profile of individual components at a detailed level.
Use a tool to generate load with the characteristics of our estimate. Our tool of choice for this is JMeter. We have also used SAAS tools like Loadimpact to avoid the effort of setting up load-generating hardware. It's important that the load generator correlates to the estimated load. Don’t go too small or too high. We usually try 2-3X of the expected load. One best practice is also to keep increasing the load to know the breaking points for future planning.
Profile:Once you have your load suite running and you find bottlenecks in the system, please don’t try to fix it. Though it sounds common sense, often we have seen developers thinking of a clever idea and implementing it which might improve the performance of the component by 20% but not make any dent at all in the overall performance. Many times this is because the component’s contribution will be very small to the overall time taken.
Use profilers for each part of the application and spend enough time on profiling before attempting to fix. Our experience is that on a slow request typically 80-90% will go in a particular part of the request. May be a slow query or slow DOM binding for instance.
Our tools of choice for this:
- FrontEnd - Google Chrome developer tools, PageSpeedInsight, Phantomas
- PHP layer - Blackfire, Symfony profiler
- Database - Depends on the database. For instance, for MySQL, we use the inbuilt profiler and Jet profiler.
Implement/Improve:There are a lot of strategies for improving each part of a complex application. Parts being frontend, Symfony / App layer and database.
FrontEnd:
Page speed is very important for the user’s experience and is often cited as the most important UX element these days. On top of it, if your application needs to be SEO friendly, then page speed is important as Google uses it as one of the variables to compute SEO rank.
There are a lot of things that can be done to improve page speed (independent of the application speed).
In any modern application, a large part of the user’s perceived performance is because of the time taken to get the data/assets from the server and render the UI. A typical profile of a request will look like
Make sure your application gets an A on all metrics tracked by YSlow. Google maintains a nice set of rules for this:https://developers.google.com/speed/docs/insights/rules.
Your biggest bang for the buck in terms of perceived performance will come from frontend tuning rather than Symfony/Appserver.
Database layer:
We will not go into details on what all we can do in the DB layer as it is a series of blogs in itself. But some pointers.
Slow query log:Enable slow query log and make sure none of your queries do a full table scan and return within a threshold we have defined. Many times, teams will clear the slow query log and then forget about it. Then in one release, a developer will check in a nasty join with full table scan on a couple of big tables which will drag the whole application down. So set up a process as part of your QA to verify this on every release. If you are deploying an NPM tool like New Relic or AppDynamics, there are more sophisticated ways.
Many times, in spite of all right indices, you will find full table scans in slow query log. We will give one example, but there are many such cases.
# Time: 130816 11:40:12
# User@Host: root[root] @ localhost [127.0.0.1]
# Query_time: 7.168569 Lock_time: 0.000211 Rows_sent: 100000 Rows_examined: 200000
SET timestamp=1376633412;
select schedule0_.id as col_0_0_,
schedule0_.event_name as col_1_0_,
schedule0_.description as col_2_0_,
schedule0_.location as col_3_0_,
schedule0_.event_type as col_4_0_,
schedule0_.start_date as col_5_0_,
schedule0_.end_date as col_6_0_,
schedule0_.start_time as col_7_0_,
schedule0_.end_time as col_8_0_,
schedule0_.remind_before as col_9_0_,
schedule0_.remind_time_type as col_10_0_,
schedule0_.repeat_every as col_11_0_,
schedule0_.sunday as col_12_0_,
schedule0_.monday as col_13_0_,
schedule0_.tuesday as col_14_0_,
schedule0_.wednesday as col_15_0_,
schedule0_.thursday as col_16_0_,
schedule0_.friday as col_17_0_,
schedule0_.saturday as col_18_0_,
user2_.id as col_19_0_ from ETEC_SCHEDULE
schedule0_ inner join ETEC_EVENT_USERS
users1_ on schedule0_.id=users1_.event_id
inner join ETEC_USER user2_ on users1_.user_id=user2_.id where
(end_date between '2013-07-28' and '2013-09-08' or
schedule0_.end_date >'2013-09-08') and user2_.id='admin';
In this instance it turned out to be because of the comparison to date in the where clause.
Just changing the where clause to fixed it.
schedule0_.start_date<='2013-09-08' AND schedule0_.end_date>='2013-07-28'
Polygot persistence:Start with a database that is a good fit for most of your use-cases. Often this is a good old RDBMS. Then for specific use-cases which have different characteristics, choose a different DB. For instance, in one of our social apps, the table that tracked cumulative user activity grew by millions per day. We moved this table alone to Cassandra.
In most of our applications, we deal with 2-3 databases.
Scale horizontally:This is again a much discussed topic but there is only so much you can do to tune a database. From the beginning, follow a share nothing architecture to help you scale horizontally when the need arises.
Symfony/App layer:
Take care of the steps called out in Symfony documentation like byte code cache: https://symfony.com/doc/current/performance.html. One thing to note here that is different from what this documentation calls out is that the future of APC seems unstable. Consider using an alternative like Redis.
And do the basic things like upgrading PHP. Performance gains in higher versions of PHP are quite high. For instance, just the difference between PHP 5.4 and 5.3 is quite measurable.
Avoid the n+1 query problem:Often we will retrieve an object and an association to serve a page. One easy way of doing it is using Doctrine’s findAll method. For instance, to display a category and its products on an eCommerce site:{% for category in products %}
<product>
<h2>{{ product.description }}</h2>
<p>Product: {{product.category.title }} {{ product. category.description }} created at {{ product.createdAt | date('d-m-Y H:i') }}</p>
</product>
{% endfor %}
But if you check the Symfony profiler, there will be one query for the category and one query for each product in that category.
Instead, do an eager fetch as below.
public function findAllCategoriesAndProducts ()
{
$qb = $this->createQueryBuilder(‘p’);
$qb->addSelect(‘x’)
->innerJoin(‘c.product’, ‘x’);
return $qb->getQuery()->getResult();
}
Bulk updating entities:This is a simple one, but novice developers of ORMs often do this to update an attribute in multiple entities:$newExpiryAt = new DateTime();
$product = $this->getDoctrine()->getRepository(‘AcmeDemoBundle:Product’)->findAll();
/** @var Product $product */
foreach ($products as $product) {
$product->setExpiryAt($newExpiryAt);
}
$this->getDoctrine()->getManager()->flush();
Instead bulk update using:
{
$qb = $this->createQueryBuilder(‘p’);
$qb->update()
->set(‘p.expiryAt’, ‘:newExpiryAt’)
->setParameter(‘newExpiryAt’, $newExpiryAt);
return $qb->getQuery()->execute();
}
Don’t bind to a container and then query for the required service. Instead, inject the service directly. If you want to know all the available services in a large code base, you can do it by
Php app / console debug: container
Use a gateway cache to improve performance and bring down the load on the app server.
Cache actions where you can:In our experience, the biggest performance impact is often achieved by proper caching.We already saw caching assets in the browser cache, usage of CDN, etc in the front-end section.In the app server layer, you can cache entire pages or parts of pages (fragments), and service-level data. This drastically brings down the response time of the request. In addition, it brings down the overall load on the system so that even cache misses are served faster.Caching pages or fragments:Entire pages or part of pages can be cached with Vanish + ESI. Even in a dynamic application, there are views which is suitable for caching and need not be accurate with real-time information. For instance, comments on blogs or recommendations. A good blog on caching fundamentals: https://blog.bytebytego.com/p/a-crash-course-in-caching-part-1
Though Symfony has an inbuilt proxy, always use a purpose-built reverse proxy like Varnish. Varnish consistently outperforms Symfony’s inbuilt proxy. https://www.symfony.fi/entry/symfony-benchmarks-symfony-proxy-vs-varnish
Caching service data:This can be done either at the ORM level by caching query results etc or/and more higher level caches. Often times we end up using 2 or more cache layers – a cache chaining pattern.Few Symfony-Specific caching steps:
- Enable query cache (this is different from query result cache). This converts the DQL query into its SQL counterpart. You would think it would be enabled by default, but its not.
- Enable metadata cache. This caches the parse metadata from annotations or XML configurations instead of doing it on every request.
- Cache query results for often requested queries on slow-changing data. You can cache raw SQL result data or explicitly cache the hydrated object.
- For really demanding applications, side steps the ORM, directly get data as associative arrays using Doctrine DBAL, and construct the object graph in the App layer.
- Though declarative object graphs are nice from a design perspective, they are performance nightmares for ORMs. So use one-to-many and many-to-many associations only as needed. And almost always avoid bi-directional associations. If you do use a lot of associations, consider doing multi-step hydration.
Are you looking to build a great product or service? Do you foresee technical challenges? If you answered yes to the above questions, then you must talk to us. We are a world-class custom .NET development company. We take up projects that are in our area of expertise. We know what we are good at and more importantly what we are not. We carefully choose projects where we strongly believe that we can add value. And not just in engineering but also in terms of how well we understand the domain. Book a free consultation with us today. Let’s work together.
.png)
.png)
.webp)
