Zend Framework: Zend_Application_Resource_Doctrine Component Proposal
| Proposed Component Name | Zend_Application_Resource_Doctrine |
|---|---|
| Developer Notes | http://framework.zend.com/wiki/display/ZFDEV/Zend_Application_Resource_Doctrine |
| Proposers | Matthew Lurz |
| Zend Liaison | TBD |
| Revision | 0.1 - 1 June 2009: Initial Draft. (wiki revision: 16) |
Table of Contents
1. Overview
Zend_Application_Resource_Doctrine is a resource plugin for the Doctrine O/RM that supports manager and connection level configuration and initialization.
A working prototype is available here in Parables/Application/Resource/Doctrine.php
2. References
- Doctrine
- Zend_Application_Resource
3. Component Requirements, Constraints, and Acceptance Criteria
- This component will allow configuration of manager level attributes.
- This component will allow configuration of connection level attributes.
- This component will allow configuration of and initialization for connections.
- This component will allow configuration of connection level event listeners.
- This component will allow configuration of manager and connection level query and result caching.
- This component will support both non-namespaced, Doctrine generated models and custom, namespaced models.
- This component does not currently support Doctrine extensions due to autoloading issues.
- This component does not currently support bitmasks for manager and connection attribute values
4. Dependencies on Other Framework Components
- Zend_Application_Resource_ResourceAbstract
5. Theory of Operation
The component will work like other resource plugins. Users will specify configuration options in an application configuration files and use Zend_Application_Bootstrap to load the resource.
6. Milestones / Tasks
- Milestone 1: [DONE] Initial proposal published for review.
- Milestone 2: Working prototype checked into the incubator supporting use cases #1, #2, ...
- Milestone 3: Working prototype checked into the incubator supporting use cases #3 and #4.
- Milestone 4: Unit tests exist, work, and are checked into SVN.
- Milestone 5: Initial documentation exists.
7. Class Index
- Zend_Application_Resource_Doctrine
8. Use Cases
| UC-01 |
|---|
Initial Setup
The two most common usage scenarios are those of generating model classes from schema and creating models by hand. In either case:
- Add a key to application.ini for registering the Doctrine namespace with the default autoloader.
If generating model classes from schema files:
- Add paths to your models and generated models anywhere the include_path is
set such as public/index.php, tests/index.php, scripts/somescript.sh, etc.
- Add a method to your bootstrap(s) for setting the fallback autoloader.
If creating model classes by hand:
- Add a method to your bootstrap(s) to configure module autoloading.
- When using table classes, you will also need to enable table classautoloading at either the manager or connection level in as:
.. or ..
| UC-02 |
|---|
Configure DSNs
Connection DSNs can be specified in any combination of string and/or array
formats.
Configure a DSN (string form)
Configure a DSN (array form)
| UC-03 |
|---|
Configure Manager Attributes
| UC-04 |
|---|
Configure Connection Attributes
39 Comments
comments.show.hideJun 02, 2009
Jochen Bünnagel
Could the proposal be modified to support modular applications? Our model files are in application/modules/MyModule/Models/ and application/modules/MyModule/Models/generated/.
Since we potentially have lots of modules I'd hate to pollute my include_path with lots of model directories. Off the top of my head I have no idea for solving this, but maybe you do?
BTW: We use Zend Framework/Doctrine naming conventions for our Model classes to be able to use the autoloader, ie. application/modules/MyModule/Models/User.php holds class MyModule_Models_User. This way we only need to put application/modules/ in the include_path. We haven't found a solution for the Doctrine generated classes yet, because we can't tell Doctrine to name the classes after its own naming scheme.
Jun 02, 2009
Matt L
Thanks for the comment.
I hesitated to attempt to solve this problem without additional input as it doesn't seem to be an easy one. Instead, I wanted to put this out there as a solution for non-modular models in hopes of obtaining feedback from the community as to how to handle more complex scenarios.
The other primary issue that I am aware of is the potential size of application.ini since so many options are available.
Oct 29, 2009
Bas K
wouldn' it be an idea to keep the size of application.ini reasonable if you could specify a seperate ini for doctrine settings.
in application.ini
in Zend_Application_Resource_Doctrine
Oct 29, 2009
Matt L
I think this a special case since this resource plugin offers so many more configuration options than others (at least the ones I've seen). I've been torn on the issue. I know quite a few users would like to use a separate config file, but this breaks with convention and I don't know how keen the ZF developers and community would be about that. Also, in some cases, the configuration is quite simple and there's really no need for a separate config file. I'd be more than happy to implement such a solution (or you could help me
if I could get feedback from the ZF developers as to whether they want to promote this practice for potentially complex resource plugins.
Oct 30, 2009
Bas K
Other resource configurations might need something similar too. Feels it would be a better fit as a feature of Zend_Application. Otherwise I'll be happy to suggest an implementation for Parables...
Nov 18, 2010
Mihail
I fully agree with the author of the idea.
Nov 19, 2010
Bas K
as it turns out zend application already supports putting 'domain' configs in seperate files.
Just do in your application.ini
config[] = some/path/to/whatever.ini
config[] = some/path/to/whatever-butnotthesame.ini
config[] = some/path/to/yetanotherconfigfile.yaml
so you could use in application.ini
config[] = some/path/to/doctrine.ini
and out inside that file all things doctrine
resources.doctrine.prop1 = value1
resources.doctrine.prop2 = value2
Jun 05, 2009
Matt L
Hi Jochen, I'm very keen on solving this problem and so will post to the Doctrine group to try to obtain some feedback. I'll let you know what I find out.
Jun 05, 2009
Matt L
Due to the potential complexity of setting up the Doctrine environment as well as connections, I'm considering/experimenting with splitting this into 2 separate resource plugins. It seems natural that the split would be between global/manager-level environment setup and connection setup. However, I don't want to make this any more complex than it needs to be. Does anyone have comments or suggestions?
Jun 11, 2009
Matt L
I've split this into 3 separate classes. Doctrine.php now initializes the manager and then creates the Connections and Paths resources. I'm unsure whether this is the correct way to implement a hierarchy of resources, but it works. I've started to add unit tests to the library and am working on a solution for modular models using http://www.marc-remolt.de/en as a starting point. If I can't get this to work then I'll inquire with the Doctrine community. I believe Doctrine 2.0 will make this easier.
Aug 09, 2009
Mattijs Hoitink
Hi Matt,
is there some code available somewhere? Because I can't seem to find it.
I was experimenting with a Doctrine resource myself until I stumbled on your proposal. Maybe I can pitch in?
Mattijs
Aug 09, 2009
Matt L
Sorry Mattijs, I need to update this proposal but was hoping to get some additional feedback first.
Code is available at http://github.com/mlurz71/parables/tree under Parables/Application/Resource.
Any feedback/assistance/etc would be greatly appreciated.
Aug 09, 2009
Matt L
Also, you can take a look at the demo app (not much to look at except the config file) here:
http://github.com/mlurz71/parables-demo/tree
Thanks again for your interest in this proposal.
Aug 22, 2009
John John
Already using it in my project, works perfectly, no bugs as yet... Looking forward to see it in trunk. Thank you very much
Aug 22, 2009
Matt L
Thanks for your interest in this proposal. It's far from perfect, but starting to come along. The simple use cases should be more or less covered at this time but there are still some nasty edge cases (mostly related to autoloading) that have yet to be worked out. Hopefully we can work out those issues in the near future. Thanks again and let me know if you run into any trouble.
Aug 23, 2009
John John
Today tried to set settings like in UC-6:
resources.doctrine.manager.auto_accessor_override = 0
resources.doctrine.manager.auto_free_query_objects = 1
resources.doctrine.manager.autoload_table_classes = 0
but they don't set up. This feature isn't implemented yet or I am using an outdated version?
Aug 23, 2009
Matt L
Sorry, the proposal needs to be updated. I intend to do so asap.
If you're working with any recent revision you just need to specify these as:
resources.doctrine.manager.attributes.attr_auto_accessor_override = 0
resources.doctrine.manager.attributes.attr_auto_free_query_objects = 1
resources.doctrine.manager.attributes.attr_autoload_table_classes = 0
See application.ini in parables-demo for a full example. Attribute naming changed due to a change in Doctrine 1.2. From the upgrade doc:
The string support of `Doctrine_Configurable::getAttribute()` and
`Doctrine_Configurable::setAttribute()` have been removed.
The reason is that the performance of this simple convenience feature is
significant and it is totally unneeded.
The following code is no longer supported.
$connection->setAttribute('use_native_enum', true);
You must now always use the following code.
$connection->setAttribute(Doctrine::ATTR_USE_NATIVE_ENUM, true);
Let me know if you run into any other issues.
Aug 24, 2009
Matt L
For the interested..
Support for modular models has been added, but is 'experimental' to say the least. Use it at your own risk. Improved support for modular models is one of, if not the issue that will be receiving the most attention in the near future.
Also, I've created a couple of initial prototype convenience scripts for rebuilding and migrating of modular models in the scripts directory of the parables-demo here.
Unfortunately, the problem of using Doctrine's extensions has yet to be solved. If anyone has had any luck using this feature of Doctrine, please contact me.
Nov 08, 2009
Benjamin Eberlei
Can you explain a bit more how that works?
Nov 08, 2009
Matt L
Sorry Benjamin, the script for generating modular models stopped working at some point and I've been unable to get it working again. You can still have a modular app/models, but the models have to be created by hand.
Sep 16, 2009
exceptione
I would like to add that a firebug profiler for doctrine has already be implemented: http://taat.pl/article/zend_framework_tutorial/step5/
// This proposal originated from: http://www.nabble.com/RFC:-ZendX_Doctrine-td23454552i20.html
Sep 26, 2009
Cristian Bichis
I would say that modular support is a must to.
I would say that standard (non-modular) ZF applications are mainly made by newbies which don't even heard about Doctrine and don't need it for their light usage.
Doctrine is more for more complex tasks so more suitable for modular applications.
So actually i think if Doctrine support for non-modular applications won't exist won't be any problem, the "target" should be for modular apps...
Sep 26, 2009
Matt L
Thanks for your comments and interest Christian.
While I agree that modular support is a must in order for this component to be useful, I don't agree that:
1) Doctrine is only useful for complex applications. On the contrary, the benefits of using Doctrine can be harnessed in even the most simple of scenarios.
2) Non-modular apps are just for noobs. Having developed both modular and non-modular applications, it seems to me that this has more to do with the complexity required by the system.
Having said that, I have implemented 'basic' modular support for this component. I use the term 'basic' as there are many potential edge cases that I have yet to explore. Take a look at http://github.com/mlurz71/parables-demo if you're interested. I welcome any and all feedback.
Thanks again.
Nov 08, 2009
Benjamin Eberlei
Imho it should be possible to configure which "model" comes into which module, the code should be generated into the respective directories.
Maybe it is necessary to extend the Doctrine Tasks to allow this. Additionally I think the "generated" folder should be renamed in Base and the base models should be generated as <Module>Model_Base<Model>.
From a look over Doctrine_Import and Doctrine_Import_Builder this is perfectly possible with a few tweaks and juozas has already written code for this we could use as a first start.
A change of the import scripts from the Doctrine CLI could be done within a Zend Tool + Doctrine support.
Nov 08, 2009
Matt L
Thanks for the input Benjamin. I didn't dig into Doctrine_Import but agree that a Zend_Tool integration point would be a better place to handle this.
Nov 08, 2009
A.J. Brown
Regarding the separate configuration issue, I think that it warrants creating a separate proposal. With the unlimited potential of resources created in the future, it would make sense to have some convention that allows for resources to have separate configuration files.
Nov 08, 2009
Matt L
Thanks for the input AJ. I agree that this issue would be best handled in a separate proposal. A similar/related proposal for module configurations has been submitted http://framework.zend.com/wiki/pages/viewpage.action?pageId=16023853, but I have yet to review.
Jan 11, 2010
Benjamin Steininger
Support for using a compiled version of Doctrine would be nice, an option with the name to the compiled version which then get's automatically required before all the other stuff is done.
A way to set the charset for the connection would be nice, but a way which gets automatically run AFTER connect and doesn't start the connect ( like a $conn->setCharset('utf8'); would do) so no unnecessary connection get's established and for example for use-cases when using the bootstrap together with the Doctrine-CLI to create the database/models and stuff (Think that could be important for Zend_Tool-Integration too)
Because if the database doesn't exist yet and the connection get's opened for running for example the "SET NAMES 'UTF-8' query, Doctrine throws an exception that it couldn't connect to the database. (Or because of an error in the records/yaml-files the database get's dropped but not recreated, so on the next try after changing records/yaml, the connection fails)
A way working for all DBMS would be using an EventListener (Doctrine_EventListener_Interface) which uses $connection->setCharset();
in the postConnect()-method, like:
class Zend_Doctrine_EventLister_Connect implements Doctrine_EventListener_Interface { /** * @var string */ protected $_charset = 'utf-8'; /** * * @param string $charset */ public function __construct($charset = 'utf-8') { $this->_charset = $charset; } /** * * @param Doctrine_Event $event */ public function postConnect(Doctrine_Event $event) { $event->getInvoker()->setCharset($this->_charset); } }Jan 12, 2010
Matt L
Thanks for the feedback Benjamin. I haven't tried using a compiled version of Doctrine and so I imagine there are issues. Unfortunately, I don't have time to work on this at the moment. Please feel free to submit a patch to the code on github, http://github.com/mlurz71/parables/tree. Thanks again.
Jan 12, 2010
Benjamin Steininger
Okay, you can look here: http://github.com/robo47/parables
Had to change a bit of the structure of the unittests and added a TestHelper for setting include-path and autoloader to get it working for me.
Features:
Unittests:
The current way I used for throwing an exception if a cache-driver or listener is not found is a bit ugly and doesn't use Zend_Loader_Autoloader, because the include always triggered an error (before it even throws it's exception) which was not really nice [a include-path-aware file_exists in the autoloader before the include(_once) would be nice].
Probably anybody has a better idea how to handle that ? Or just ignore the error because an exception is always to be thrown afterwards ?
Some thoughts and questions I had:
1) One thing in the complete design of the resource I discovered and wasn't aware of before, which in my opionion should not be as it currently is:
The methods setConnections/setPathes/setManagers contain all the logic of the resource and are called directly after the creation of the resource (automatically from the Zend_Application_Resource_ResourceAbstract::setOptions()), not after calling the init()-method like it is handled by all the other resources in the Zend Framework. Which means the exceptions are thrown on creation too, not on init.
I looked into the other Zend-Framework resources in zf 1.9.7/1.10alpah1 and they all are doing this in init() which in my opinion should be the right way.
I thing this behaviour should be changed to be conform with the other resources.
2) Is there a need to check everything which is checked in the Doctrine-Classes too ?
Instead of checking for wrong dsn, tablename, is_dir [in setPathes] and stuff, let the Doctrine-Classes throw their exceptions and rethrow them in the resource as previous-parameter of a Zend_Application_Resource_Exception.
I think that would save a lot of checks which are otherweise done EACH time and are repeated in the Doctrine-Classes afterwards each time too (which is a waste of time and io-resources) in most cases.
3) (Only important if you want to keep checking and throwing the exceptions like mentioned in 2) )
The 6 exceptions thrown in _getCache for the db-driver, could be stripped down to 3 by using only if(empty()) and don't do the array_key_exists, since empty() does an isset() itself, I don't think it is necessary to differ between undefined and empty, just three exceptions with messages like
missing options for cache-driver: db
missing option 'dsn' for cache-driver: db
missing option 'tableName' for cache-driver: db
or something like that should be enough. Same thing can be done to the memcache-part and the dsn-option in setConnections() and throw one instead of 2 possible exceptions.
4) With the compiled version I got some problems which I think are tied to the design that all the stuff is run after creation of the object not in init() because i wanted to test for compiled version in init().
The problem is, if the compiled version is used, the zend_autoloader can't load the Doctrine-classes. so if the Doctrine.compiled.php isn't loaded before setting up the resource, it want's to include all needed classes like Doctrine_Manager and that fails because it can't find them.
One way could be to completely change the resource and split it into two resources.
The first one can check if a compiled version should be used and include it and then create an instance of the second one which handles creation of manager, connection, caches and stuff.
But I have to look into this again since I didn't check if changing the setup to be done in init() would fix the problem.
5) Another question will be, which versions of Doctrine needs to be supported ? 1.0+ ? 1.1+? or only 1.2+.
] , make the application ready for a 1.3/1.4 which will probably drop Doctrine.php
(There isn't any answer in http://framework.zend.com/wiki/display/ZFDEV/Doctrine+Integration+Todo to that question yet).
Because when using only 1.2 there is no need for different tests for the attributes and no more need to use the class Doctrine [as of 1.2 the class Doctrine is deprecated and not used anymore inside Doctrine. It only inherits from Doctrine_Core to not break backwards-compatibility yet: http://www.doctrine-project.org/jira/browse/DC-91 ).
That would save 1 included File [I don't include it anymore in any Project because it is so ugly outside of it's folder
And saves all users using 1.2 the need of deploying Doctrine.php which doesn't play nice with svn:externals and Autoloaders
Jan 13, 2010
Matt L
Thanks again Benjamin. I sincerely appreciate the time you have taken to review and improve upon this proposal.
Unfortunately, I'm not going to have a chance to give the issues you raised my full attention for a couple of weeks. If you don't mind a delay, I will review and reply asap.
With regard to Doctrine minor versions supported, as I recall I left off attempting to support all minor versions in the 1.x series but think this decision should be revisited.
Jan 13, 2010
Benjamin Steininger
No problem
I will keep on testing/extending/posting here and you can review the stuff when you have time.
Jan 25, 2010
Matt L
Hey Benjamin..I think I might have some time over the next couple of weeks to work on this to resolve the issues you've raised. Let me know if you have time and if so how you would like to proceed.
Jan 25, 2010
Benjamin Eberlei
Hey Matt,
for the Zend_Tool integration i already took your resource and did some little refactorings, for example you instantiate the doctrine manager and connections in the set*() routines already, ahlthough this has to be delayed until ->bootstrap('Doctrine') is called.
Also i changed the return value of the Resource init() to return a "Zend_Doctrine_Registry" that has the connections, manager and some of the Tool relevant options like paths and generate model options available.
I will commit the code to my zf svn user branch tonight.
Jan 25, 2010
Matt L
Thank you. Ignorance is my only excuse for performing initialization of the resource outside of init. Sorry to all who were left scratching their heads. I'm wondering how we proceed with regard to repos. I don't want to make things any more complex than they need to be and have no particular attachment to my git repo if there is a better way to work on this together. It would also be nice to get all of the apples in 1 basket, so to speak, so that Doctrine 1.x and 2.x related integrated points can be had from a single repo. Thoughts?
Apr 02, 2010
Benjamin Steininger
Anything new here ? have been about 2 month again.
I send in a CLA yesterday which was accepted already so my code can be used. (didn't think about that yet)
Apr 02, 2010
Matt L
Nothing new to my knowledge. The last I knew both yourself and Benjamin Eberlei had made major improvements. Personally, I'd like to see those changes integrated into or replace this proposal. Doctrine 1.2 is going to be maintained until 2011-06-01 and so I think it's still worth the effort if anyone can find the time.
Apr 02, 2010
Benjamin Eberlei
I had to do some changes to Doctrine 1.2 to allow integration with Zend modular structure, this was released with 1.2.2. My code for the Doctrine resource in my user-branch, the tool code is there too:
http://framework.zend.com/svn/framework/standard/branches/user/beberlei/zfdoctrine/library/Zend/Doctrine/
Matthew Weierophinney told me he doesn't know if there will be a 1.11 or not yet, this will be decided in the next weeks. If there will be a 1.11 release then I think Doctrine should be in there. If there is no 1.11 i think we don't need Doctrine 1 support then for 2.0, probably makes no sense.
Apr 02, 2010
Matt L
Thanks for the update and improvements Benjamin. If there isn't a 1.11 then at least your work will be available to those still using a 1.x release. Personally, I'm excited to take a look and to see how the Zend_Tool integration was done. Thanks again!