Klaus Guenther [2004-04-27 18:53 UTC] I am very much in support of this RFC. Hopefully it will boost the general acceptance of PEAR. This provides a much more definite and detailed list of duties/abilities for the QA team than was provided for the Group. The most important thing is to get going. It's not worth sitting on our hands and letting people trash PEAR for not going anywhere. So let's go! :-)
Greg Beaver [2004-04-29 03:03 UTC] A serious omission from this RFC involves regression testing.
The QA team should actively enforce regression testing in a format that is both easy to use and easy to set up.
This means that either PHPUnit, PHPUnit2, or Simpletest should be recommended for object-based code, and run-tests should be recommended for code that can only be run from the global context. The only example I can think of that requires global context is code that relies upon the output of debug_backtrace(), although there may be other examples.
The QA team should also actively define standard ways to structure regression tests.
Regression tests should consist of 2 kinds of test suites
- API tests
- internal functionality tests.
The API test suites should consist of bounds checking for every public method of a class, and testing expected output. These tests should work for every minor version without modification.
The internal functionality tests consist of bug-fixing tests, and other verification of functionality that may break on a minor version release with the shifting of private functionality.
Any package that does NOT require external setup to function must have an API test suite to be considered stable. Packages that require external setup, such as database or internet connections, should be handled on a case-by-case basis.
The pear run-tests command shall be modified to work with PHPUnit/PHPUnit2 test suites in a future version of PEAR to make it easy to run regression tests.
Helgi Þormar Þorbjörnsson [2004-04-29 20:40 UTC] Greg: After discussing this with Lukas we decided that we weren't going to add this to the RFC, since the RFC is big enough, and this should be done in a separated RFC, i.e. your idea.
This isn't really a QA matter, but it would help the QA in some situation but this isn't a must for QA, but is a nice addition if someone (Greg?) wants to make a RFC for this :)
|
