Many times, there is a knock on Symfony framework that is slow. Frequently, you will hear quips like “Symfony has a great set of features for building enterprise applications but performs worse than <insert your framework here>”. There are also many benchmarks that will take a trivial use case and an untuned Symfony and paint a sorry picture.
But we have rolled out pretty zippy applications that handle tons of loads using Symfony. We will share a few tips and tricks that we have used in our apps.
Performance as a Design Consideration
First off, performance is an important aspect and cannot be an afterthought. Often, teams will finish the functionality and then do a 'performance testing and fixing' phase. It goes without saying that this is the wrong approach.The overall performance of the application is the culmination of many design decisions and iimplementation tricks, and hence performance should be an exercise throughout the life cycle of the project.
Our Framework for Performance and Scalability
Given that at Ideas2IT, we offer performance and scalability as a service offering itself and have invested in a partnership with AppDynamics, we have developed a framework that follows the cycle of Estimate, Measure, Profile, Implement/Improve – Repeat, throughout the life cycle of a project to achieve desirable performance and scalability.
Estimation Phase:
It is important to get an estimate of what is expected of the system in terms of
- Load
- Data Volume
- Availability
- Cost constraints
If you don't do this exercise, we will under-tune the system. And they sometimes might overplan which can be a costly exercise.
Measurement Phase:
Never attempt to fix performance without measuring the performance profile of individual components at a detailed level.
Use a tool to generate a load with the characteristics of our estimate. Our tool of choice for this is JMeter. We have also used SAAS tools like Loadimpact to avoid the effort of setting up load-generating hardware. It's important that the load generator correspond the estimated load. Don’t go too small or too high. We usually try 2-3X of the expected load. One best practice is also to keep increasing the load to know the breaking points for future planning.
Profile:Once you have your load suite running and you find bottlenecks in the system, please don’t try to fix it. Though it sounds common sense, often we have seen developers thinking of a clever idea and implementing it, which might improve the performance of the component by 20% but not make any dent at all in the overall performance. Many times, this is because the component’s contribution will be very small to the overall time taken.
Use profilers for each part of the application, and spend enough time on Profiling Phase before attempting to fix. Our experience is that on a slow request, typically 80-90% will go into a particular part of the request. It may be a slow query or slow DOM binding for instance.
Our tools of choice for this:
- FrontEnd - Google Chrome developer tools, PageSpeedInsight, Phantomas
- PHP layer - Blackfire, Symfony profiler
- Database - Depends on the database. For instance, for MySQL, we use the inbuilt profiler and Jet profiler.
Implementation and Improvement Phase:
There are a lot of strategies for improving each part of a complex application. Parts are the frontend, Symfony / App layer and database.
FrontEnd Optimization:
Page speed is very important for the user’s experience and is often cited as the most important UX element these days. On top of that, if your application needs to be SEO-friendly, then page speed is important, as Google uses it as one of the variables to compute SEO rank.
There are a lot of things that can be done to improve page speed (independent of the application speed).
In any modern application, a large part of the user’s perceived performance is because of the time taken to get the data/assets from the server and render the UI. A typical profile of a request will look like
Make sure your application gets an A on all metrics tracked by YSlow. Google maintains a nice set of rules for this: Google maintains a nice set of rules for Google Speed Rules.
Your biggest bang for the buck in terms of perceived performance will come from frontend tuning rather than Symfony/Appserver.
Database Layer Optimization
We will not go into details on what all we can do in the DB layer, as it is a series of blogs in itself. But some pointers.
Slow query log:
Enable slow query log and make sure none of your queries do a full table scan and return within a threshold we have defined. Many times, teams will clear the slow query log and then forget about it. Then, in one release, a developer will check in a nasty join with a full table scan on a couple of big tables which will drag the whole application down. So set up a process as part of your QA to verify this on every release. If you are deploying an NPM tool like New Relic or AppDynamics, there are more sophisticated ways.
# Time: 130816 11:40:12
# User@Host: root[root] @ localhost [127.0.0.1]
# Query_time: 7.168569 Lock_time: 0.000211 Rows_sent: 100000 Rows_examined: 200000
SET timestamp=1376633412;
select schedule0_.id as col_0_0_,
schedule0_.event_name as col_1_0_,
schedule0_.description as col_2_0_,
schedule0_.location as col_3_0_,
schedule0_.event_type as col_4_0_,
schedule0_.start_date as col_5_0_,
schedule0_.end_date as col_6_0_,
schedule0_.start_time as col_7_0_,
schedule0_.end_time as col_8_0_,
schedule0_.remind_before as col_9_0_,
schedule0_.remind_time_type as col_10_0_,
schedule0_.repeat_every as col_11_0_,
schedule0_.sunday as col_12_0_,
schedule0_.monday as col_13_0_,
schedule0_.tuesday as col_14_0_,
schedule0_.wednesday as col_15_0_,
schedule0_.thursday as col_16_0_,
schedule0_.friday as col_17_0_,
schedule0_.saturday as col_18_0_,
user2_.id as col_19_0_ from ETEC_SCHEDULE
schedule0_ inner join ETEC_EVENT_USERS
users1_ on schedule0_.id=users1_.event_id
inner join ETEC_USER user2_ on users1_.user_id=user2_.id where
(end_date between '2013-07-28' and '2013-09-08' or
schedule0_.end_date >'2013-09-08') and user2_.id='admin';
Fixed Where Clause:
Example of Full Table Scan Issue
In this instance, it turned out to be because of the comparison to date in the where clause.
# Time: 130816 11:40:12
# User@Host: root[root] @ localhost [127.0.0.1]
# Query_time: 7.168569 Lock_time: 0.000211 Rows_sent: 100000 Rows_examined: 200000
SET timestamp=1376633412;
SELECT schedule0_.id AS col_0_0_,
schedule0_.event_name AS col_1_0_,
schedule0_.description AS col_2_0_,
schedule0_.location AS col_3_0_,
schedule0_.event_type AS col_4_0_,
schedule0_.start_date AS col_5_0_,
schedule0_.end_date AS col_6_0_,
schedule0_.start_time AS col_7_0_,
schedule0_.end_time AS col_8_0_,
schedule0_.remind_before AS col_9_0_,
schedule0_.remind_time_type AS col_10_0_,
schedule0_.repeat_every AS col_11_0_,
schedule0_.sunday AS col_12_0_,
schedule0_.monday AS col_13_0_,
schedule0_.tuesday AS col_14_0_,
schedule0_.wednesday AS col_15_0_,
schedule0_.thursday AS col_16_0_,
schedule0_.friday AS col_17_0_,
schedule0_.saturday AS col_18_0_,
user2_.id AS col_19_0_ FROM ETEC_SCHEDULE
schedule0_ INNER JOIN ETEC_EVENT_USERS
users1_ ON schedule0_.id=users1_.event_id
INNER JOIN ETEC_USER user2_ ON users1_.user_id=user2_.id WHERE
(end_date BETWEEN '2013-07-28' AND '2013-09-08' OR
schedule0_.end_date >'2013-09-08') AND user2_.id='admin';
Just changing the where clause fixed it.
schedule0_.start_date<='2013-09-08' AND schedule0_.end_date>='2013-07-28'
Polygot persistence:
Start with a database that is a good fit for most of your use cases. Often, this is a good old RDBMS. Then, for specific use cases that have different characteristics, choose a different DB. For instance, in one of our social apps, the table that tracked cumulative user activity grew by millions per day. We moved this table alone to Cassandra.
In most of our applications, we deal with 2-3 databases.
Scale horizontally:
This is again a much discussed topic, but there is only so much you can do to tune a database. From the beginning, follow a share nothing architecture to help you scale horizontally when the need arises.
Symfony/App Layer Optimization
Symfony/App Layer:
Take care of the steps called out in Symfony documentation like bytecode cach One thing to note here that is different from what this documentation calls out is that the future of APC seems unstable. Consider using an alternative, like Redis.
And do the basic things,Often, we will retrieve an object and an association to serve a page. One easy way of doing it is using Doctrine’s like upgrading PHP. Performance gains in higher versions of PHP are quite high. For instance, just the difference between PHP 5.4 and 5.3 is quite measurable.
Avoid the n+1 query problem:
Often we will retrieve an object and an association to serve a page. One easy way of doing it is using Doctrine’s findAll method. For instance, to display a category and its products on an eCommerce site:
{% for category in products %}
<product>
<h2>{{ product.description }}</h2>
<p>Product: {{product.category.title }} {{ product. category.description }} created at {{ product.createdAt | date('d-m-Y H:i') }}</p>
</product>
{% endfor %}
But if you check the Symfony profiler, there will be one query for the category and one query for each product in that category.
Eager Fetch Example:
Instead, do an eager fetch as below.
public function findAllCategoriesAndProducts ()
{
$qb = $this->createQueryBuilder(‘p’);
$qb->addSelect(‘x’)
->innerJoin(‘c.product’, ‘x’);
return $qb->getQuery()->getResult();
}
Bulk updating entities:
This is a simple one, but novice developers of ORMs often do this to update an attribute in multiple entities:
$newExpiryAt = new DateTime();
$product = $this->getDoctrine()->getRepository(‘AcmeDemoBundle:Product’)->findAll();
/** @var Product $product */
foreach ($products as $product) {
$product->setExpiryAt($newExpiryAt);
}
$this->getDoctrine()->getManager()->flush();
Bulk Update Example:
Instead bulk update using:
{
$qb = $this->createQueryBuilder(‘p’);
$qb->update()
->set(‘p.expiryAt’, ‘:newExpiryAt’)
->setParameter(‘newExpiryAt’, $newExpiryAt);
return $qb->getQuery()->execute();
}
Service Injection:
Don’t bind to a container and then query for the required service. Instead, inject the service directly. If you want to know all the available services in a large code base, you can do it by
Php app / console debug: container
Gateway Cache:
Use a gateway cache to improve performance and bring down the load on the app server.
Caching Strategies
Cache actions where you can:
In our experience, the biggest performance impact is often achieved by proper caching.
We already saw caching assets in the browser cache, usage of CDN, etc in the front-end section.In the app server layer, you can cache entire pages or parts of pages (fragments), and service-level data. This drastically reduces the response time to the request. In addition, it brings down the overall load on the system so that even cache misses are served faster.
Caching pages or fragments:
Entire pages or part of pages can be cached with Vanish + ESI. Even in a dynamic application, there are views that are suitable for caching and need not be accurate with real-time information. For instance, comments on blogs or recommendations. A good blog on caching fundamentals: A good blog on Caching Fundamentals.
Though Symfony has an inbuilt proxy, always use a purpose-built reverse proxy like Varnish.
Caching service data:
This can be done either at the ORM level by caching results, etc., or/and more higher level caches. Oftentimes we end up using 2 or more cache layers – a cache chaining pattern.
Few Symfony-Specific caching steps
- Enable Query Cache: This is different from query result cache. This converts the DQL query into its SQL counterpart. You would think it would be enabled by default, but it’s not.
- Enable Metadata Cache: This caches the parsed metadata from annotations or XML configurations instead of doing it on every request.
- Cache Query Results: Cache query results for often requested queries on slow-changing data. You can cache raw SQL result data or explicitly cache the hydrated object.
- Direct Data Access: For really demanding applications, side-step the ORM, directly get data as associative arrays using Doctrine DBAL, and construct the object graph in the App layer.
- Use Associations Wisely: Though declarative object graphs are nice from a design perspective, they are performance nightmares for ORMs. So use one-to-many and many-to-many associations only as needed. And almost always avoid bi-directional associations. If you do use a lot of associations, consider doing multi-step hydration.
%20How%20Generative%20AI%20Is%20Revolutionizing%20Customer%20Experience.png)
%20Leading%20LLM%20Models%20Comparison_%20What%E2%80%99s%20the%20Best%20Choice%20for%20You_.png)
%20Generative%20AI%20Strategy_%20A%20Blueprint%20for%20Business%20Success.png)
%20Mastering%20LLM%20Optimization_%20Key%20Strategies%20for%20Enhanced%20Performance%20and%20Efficiency.png)
.png)
.png)
.png)
.webp)
