Caching and data analysis will move your Symfony2 application to the next level

Editor's Notes

• #2: OK the real title is zzz and blblabla will move your something to the next level
• #3: This is a not invented here talk, The topics I ‘ d like to discuss are all well known, I have not invented anything please go there to read the references such as the talk of Jordi etc. I’m here to share my knowledge SEE THE REFERENCES
• #4: I love the Symfony2 framework, G ave us: a breath of FRESHNESS to the whole php world , GREAT DOCUMENTATION/ NOT FRAMEWORK but BESTPRACTICES , great community / METHODOLOGIES gave as a real dark passenger, dark passenger means that there ’ s a little genious/geek that is on your shoulder when you develop B ut this dark passengere LOOK AT YOU not at the code at YOU M y dark passenger speaks FRANCE You ’ ave not Follow^ng coding standards, dependency injection. When I met this small dark passenger he grows up, I feel better my libraries are more usable. I realized that bigger is your dark passenger and the more you are a cool developer. S o at terravision we do love sf2 and it ’ s fabpot passenger.
• #5: OK the real title is zzz and blblabla will move your something to the next level
• #6: OK the real title is zzz and blblabla will move your something to the next level
• #7: We are developer our ecosystem create a lot of data.    [ok hey liuggio what problem do you have with data?] -------- data is difficult to measure, difficult to read, data is time consuming. ------- data is boring,-- data sucks * (sometimes) Logging ---------- If you think that logging is a file with a lot of unreadable information. Logging will have a Matrix effect with data. You don’t’need logging only when something bad happen How many of you use logging, How many of you thinks that logging is a file with too much iformation in it… Logging is a black hole in the developers career,
• #8: [ok hey liuggio what problem do you have with data?] data is difficult to measure, difficult to read, data is time consuming. ------- data is boring,-- data sucks * (sometimes)
• #9: what is data? different information from ----- Note riunione (5/15/13 08:26) ----- data gathers
• #10: what is data? different information from ----- Note riunione (5/15/13 08:26) ----- data gathers
• #11: I do we do, we used to ignore the most of data that we receive. But sometimes we track it, maybe passively but we keep track of some events for example logging
• #12: What do you do? 1. I ignore them 2. I ignore them but I have a clear coscience (because there ’ s a file for it) 3. I ignore them mmm but I ’ ll use regex when an halling customer will call (if you have a problem that you want to solve with the regex, you have 2 problem) I f you try to solve a problem with a regular expression, you have 2 problems. 4. I ignore the log, regex and the customer. S ometimes you need to investigate, A lways you need to be alerted when .. Y ou want to see graphes In the daylife is very important have PULL information like a big lcd monitor over there
• #13: Why should we use all that data? What is business value: all the things that produces money W e are coder, we love write code, but our code produce business, W e use unit testing in order to have maintainability of the code, this could be a business value we use function testing, because we want to add new features easily this could be a business value A nd we want that if angry customer will call you we have already the answer Y ou want a fast user experience Y ou want to know what is happening in your application --------- code is not a trophy functionality measure it in order to know face the reality measure on production
• #14: If you don’t understand this slide you are not stupid maybe because you never seen the ‘ hitchhiker's guide to the galaxy ’ movie. Hitchhikers
• #15: We use phpunit in order to easily add new feature and have code manintainability not because is cool ----- Note riunione (5/16/13 17:55) ----- each company has it's own business value.
• #16: All of this are some business value that you could achieve tracking data. Business Value is everything that produces € ( For example Unit testing produce business value (we do use unit testing in order to have maintainability of the code (easy to add a new feature) that is a business value.) Each company has its own definition of business Value, but we could define some needs that could increment your Business Value. (could because if you don ’ t care it ’ s ok) Store log and error data (investigate) Complex query on past events Easily Locate the bugs Know Servers status: healthy/resources Know Application status: Responding as it should? Know what is happening in my ecosystem Understand why and where the app. is slow Analyze the behaviour when it ‘ s running I want to watch data I want to be alerted Fast website I want to know how if after a certain event “ how much ” something else changed Interpolate data from different sources               (after the deploy, how many t-shirt were bought?) We added to many information here Investigate means that we need to collect and create relations among different logs and make queries Be alerted means that we want on different context an action Visualize means meseaure and draw from the data Noi creiamo cose  e dobbiamo interessarci del business value, il nostro codice produce business value add new features easily business value  -> all the things that produces €€ code is not a trophy functionality T here ’ s a name for everything … measure it in order to know face the reality measure on production ----- Note riunione (5/14/13 00:11) ----- satisfy ----- Note riunione (5/16/13 17:55) ----- this is business value from the developer side and data
• #17: In order to do this we need to add different actions ----- Note riunione (5/15/13 08:50) ----- perform actions
• #18: In order to do this we need to add different actions ----- Note riunione (5/16/13 11:20) ----- in here there's the talk title: from metrics and optimizing you obtain a business value
• #19: I’d like to talk with you about Metrics Measure: WE WANT TO FACE THE REALITY The terms metric and measure have some overlap. We use measure for more concrete or objective attributes and metric for more abstract, higher-level, or somewhat subjective attributes. For instance, lines of code (LOC) is a measure: it is objective and concrete The robustness is a metrics M easurment are the basis for the metrics Measures help us approximate less tangible metrics.
• #20: Metrics is a link of two worlds technical and business vision of IT.    [yes ok but what is?] ------ definition of metrics … sucks is so general and everybody has their own, ----- Metrics -- definition       metrics gives you qualitatively information (and measure quantity information) ----- Note riunione (5/15/13 09:15) ----- line of code examples measure, metrics
• #21: A quantitatively expressed reduction of uncertainty based on one or more observations. Think a measure as a quantitative number (image you could measure anything) This “ uncertainty reduction ” point of view is what is critical to busi- ness reduction can be worth millions. HOW TO MEASURE ANYTHING Hubbard measurement process of experimentally obtaining one or more quantity values that can reasonably be attributed to a quantity
• #22: When you don’t know something you could ask to them. What metrics is
• #23: Metrics number visual ops it’s a ninja territory?
• #24: Why metrics is so important? Is a join of two worlds
• #25: Metrics in order to produce Graphs → Decision decision because you could Collects and aggregates from different sources sources of metrics: webservers, application, cache server, averything! Better decision using numbers  -- @coda
• #26: How to do metrics? StatsD We want a great tool in order to add easily new metrics. From every envoirnment, every language every where.
• #28: Calculate the amout of php ram during it’s peak time Ready for the WTF effect? This is a “timer”
• #29: Each stat is in its own " bucket ” the KEY, They are not predefined anywhere. Buckets can be named anything that will translate to Graphite (periods make folders, etc) After the flush interval timeout (defined by config.flushInterval, default 10 seconds), stats are aggregated and sent to an upstream backend service.
• #30: A gauge simply indicates an arbitrary value at a point in time and is the most simple type in StatsD. It just takes any number and ships it to the backend.
• #31: Counters are simple. It adds a value to a bucket and stays in memory until the flush interval.
• #32: Timers collects numbers. They does not necessarily need to contain a value of time. You can collect bytes read, number of objects in some storage, or anything that is a number. Useful for: mean/upper/sum/lower/count Operation
• #33: Timers collects numbers. They does not necessarily need to contain a value of time. You can collect bytes read, number of objects in some storage, or anything that is a number. Useful for: mean/upper/sum/lower/count Operation
• #34: Here is a golden rule that you find in all talks about metrics I prefer metrics CAN … Because too much not usefull information could go to CONFUSION. == with Metrics you track everythings that moves  [on internet this is like  a cool definition] BUT
• #35: Se teniamo sempre alto il livello di razionalità questo ci farà dire:
• #36: È molto
• #37: Se teniamo sempre alto il livello di razionalità questo ci farà dire:
• #38: Better decision using numbers  -- @coda Good and Fast decisions help the business
• #39: Better decision using numbers  -- @coda Good and Fast decisions help the business
• #40: OK the real title is zzz and blblabla will move your something to the next level
• #42: Another topic that has a lot of usages is the cache What do you think if I say cache?
• #43: Another topic that has a lot of usages is the cache What do you think if I say cache?
• #44: Another topic that has a lot of usages is the cache What do you think if I say cache?
• #46: ----- Note riunione (5/11/13 21:43) ----- proxy
• #49: This is a great idea, We are web developer So we need to follow the web protocol Why I didn’t think about it before?
• #53: ----- Note riunione (5/14/13 00:11) ----- scrivere nei commenti fig!!
• #56: Another topic that has a lot of usages is the cache What do you think if I say cache?
• #59: OK the real title is zzz and blblabla will move your something to the next level
• #64: Y ou could do by yourself This code is problematic, and adds a lot of complexity that makes your unit tests' code even worse. https://github.com/Ocramius/ProxyManager/blob/master/docs/lazy-loading-value-holder.md https://github.com/Ocramius/symfony-docs/blob/931091d8e3aa0b9d4f1393d313388d5ee1a5d47d/components/dependency_injection/lazy_services.rst
• #66: flush multiples entities      $em->flush(array($entityA, $entityB); paginator      master-slave     Flush to Master Read from Slave join to interface (Target Entity Resolver: /**    @ORM\\ManyToOne(targetEntity="Tvision\\Bundle\\OrderBundle\\Entity\\StatusInterface") **/
• #67: flush multiples entities      $em->flush(array($entityA, $entityB); paginator      master-slave     Flush to Master Read from Slave join to interface (Target Entity Resolver: /**    @ORM\\ManyToOne(targetEntity="Tvision\\Bundle\\OrderBundle\\Entity\\StatusInterface") **/
• #68: ----- Note riunione (5/14/13 00:11) ----- levare flush
• #69: The deferred implicit policy is the default change tracking policy and the most convenient one.property-by-property comparison at commit time and also detects changes to entities or new entities that are referenced by other managed entities ( “ persistence by reachability ” ) Explicit only considers entities that have been explicitly marked for change detection through a call to EntityManager#persist(entity) or through a save cascade.
• #71: Starting from the daylife in order to undestand UOW ----- Note riunione (5/16/13 11:41) ----- there's a line that could'nt be there
• #72: T he unit of work class in doctrine ha 3206 lines, is complex UOW start with with init of EM or after a flush. The size of a Unit of Work mainly refers to the number of managed entities Is an Entity managed by UOW when is persisted or when is fetched Identity Map: In this case the Article is accessed from the entity manager twice, but modified in between. How costly a flush operation is, mainly depends on two factors: The size of the EntityManager ’ s current UnitOfWork. The configured change tracking policies You can get the size of a UnitOfWork as follows: <?php $uowSize = $em->getUnitOfWork()->size();
• #73: T he unit of work class in doctrine ha 3206 lines, is complex UOW start with with init of EM or after a flush. The size of a Unit of Work mainly refers to the number of managed entities Is an Entity managed by UOW when is persisted or when is fetched Identity Map: In this case the Article is accessed from the entity manager twice, but modified in between. How costly a flush operation is, mainly depends on two factors: The size of the EntityManager ’ s current UnitOfWork. The configured change tracking policies You can get the size of a UnitOfWork as follows: <?php $uowSize = $em->getUnitOfWork()->size();
• #74: T he unit of work class in doctrine ha 3206 lines, is complex UOW start with with init of EM or after a flush. The size of a Unit of Work mainly refers to the number of managed entities Is an Entity managed by UOW when is persisted or when is fetched Identity Map: In this case the Article is accessed from the entity manager twice, but modified in between. How costly a flush operation is, mainly depends on two factors: The size of the EntityManager ’ s current UnitOfWork. The configured change tracking policies You can get the size of a UnitOfWork as follows: <?php $uowSize = $em->getUnitOfWork()->size();
• #75: The deferred implicit policy is the default change tracking policy and the most convenient one.property-by-property comparison at commit time and also detects changes to entities or new entities that are referenced by other managed entities ( “ persistence by reachability ” ) Explicit only considers entities that have been explicitly marked for change detection through a call to EntityManager#persist(entity) or through a save cascade.
• #76: Deferred Implicit P ro : persist($entity) is not necessary Cons : bad performance with large UOW Deferred Explicit Pro : flush operation cheaper Cons : DB != UOW spread persists into the code Unit of work now has $entity.title=liuggio loves symfony2 Database has flushed without persist $entity.title=liuggio loves symfony2 Pro : $entityManager->persist($entity) is not necessary Cons : Slowness, Doctrine has to check property-by-property for all the object in the unit of work Explicit Doctrine2 tracking policy 23 April 2013 doctrine2 1 php 9 Which tracking policy are you using? With doctrine you can change the tracking policy for each entity, the tracking policy is the way in which Doctrine understands if a entity property must be ‘ saved ’ in the database. Doctrine definition: “ Change tracking is the process of determining what has changed in managed entities since the last time they were synchronized with the database. ” There are 3 tracking policies: Implicit, Explicit, Nofity. Implicit tracking (the default one) The implicit is the default one and the slowest, you don ’ t need to explicit call ‘ persist ’ eg. // $entity is not a new object, and has $entity->Title = 'liuggio loves tvision.github.io' $entity->setTitle('liuggio loves symfony2'); $entityManager->flush(); echo $entity->getTitle(); // output 'liuggio loves symfony2' Unit of work now has $entity.title=liuggio loves symfony2 Database has flushed without persist $entity.title=liuggio loves symfony2 Pro : $entityManager->persist($entity) is not necessary Cons : Slowness, Doctrine has to check property-by-property for all the object in the unit of work The explicit tracking (the suggested one) You have to explicit flag which object you want to persist, than Doctrine will check property-by-property in order to find with property has changed. Pro : Faster than implicit, must if you have a lot of entities. Cons : the object in the unit of work could be different from that one in the db, not easy to debug. The example
• #77: Temporarily mark entities as read only. If you have a very large UnitOfWork but know that a large set of entities has not changed, just mark them as read only with $entityManager->getUnitOfWork()->markReadOnly($entity).
• #79: T he unit of work class in doctrine ha 3206 lines, is complex UOW start with with init of EM or after a flush. The size of a Unit of Work mainly refers to the number of managed entities Is an Entity managed by UOW when is persisted or when is fetched Identity Map: In this case the Article is accessed from the entity manager twice, but modified in between. How costly a flush operation is, mainly depends on two factors: The size of the EntityManager ’ s current UnitOfWork. The configured change tracking policies You can get the size of a UnitOfWork as follows: <?php $uowSize = $em->getUnitOfWork()->size();
• #83: OK the real title is zzz and blblabla will move your something to the next level
• #87: I’d like to talk to you about metrics I do love metrics Mr smisth sentence:
• #88: È molto
• #90: OK the real title is zzz and blblabla will move your something to the next level
• #91: OK the real title is zzz and blblabla will move your something to the next level
• #92: OK the real title is zzz and blblabla will move your something to the next level
• #93: OK the real title is zzz and blblabla will move your something to the next level
• #94: OK the real title is zzz and blblabla will move your something to the next level
• #95: OK the real title is zzz and blblabla will move your something to the next level
• #96: OK the real title is zzz and blblabla will move your something to the next level
• #97: OK the real title is zzz and blblabla will move your something to the next level
• #98: OK the real title is zzz and blblabla will move your something to the next level