Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add DB seed and tests for coverart in API #248

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions application/database/seeds/DatabaseSeeder.php
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't sure if this was going to work for multiple test classes, but the scrubbed data has a DROP TABLE before each section, so it does actually tidy itself up nicely when the seeder reruns.

One downside of this is that, since it uses the same database for testing and local dev, it will obliterate any data that people have manually crafted to use on their machines.

That said, maybe we could combine it with this branch that hits a separate test database. (The application/config/testing/database.php config is what you'd take from that branch, and people would have to have the user/database already set up similar to what the commands in Bootstrap.php are doing, but maybe that's something we can put in a setup script or in the CONTRIBUTING guide.)

Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
<?php

class DatabaseSeeder extends Seeder {

public function run()
{
$filename = __DIR__ . '/librivox_catalog_scrubbed.sql.bz2';
$dump = fopen("compress.bzip2://$filename", 'r');
$query = '';
while (True)
{
$line = fgets($dump);
if (! $line) break;
if (str_starts_with($line, '--')) continue;
$line = trim($line, "\r\n");
$query = $query . $line;
if (str_ends_with($query, ';'))
{
$this->db->query($query);
$query = '';
}
}
fclose($dump);
}
}
Binary file not shown.
133 changes: 133 additions & 0 deletions application/libraries/Seeder.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
<?php
/**
* Part of ci-phpunit-test
*
* @author Kenji Suzuki <https://github.com/kenjis>
* @license MIT License
* @copyright 2015 Kenji Suzuki
* @link https://github.com/kenjis/ci-phpunit-test
*/

class Seeder
{
/**
* @var CI_Controller
*/
private $CI;

/**
* @var CI_DB_query_builder
*/
protected $db;

/**
* @var CI_DB_forge
*/
protected $dbforge;

/**
* @var string
*/
protected $seedPath;

/**
* @var array
*/
protected $depends = [];

public function __construct()
{
$this->CI =& get_instance();
$this->CI->load->database();
$this->CI->load->dbforge();
$this->db = $this->CI->db;
$this->dbforge = $this->CI->dbforge;
}

/**
* Run another seeder
*
* @param string $seeder Seeder classname
* @param bool $callDependencies
*/
public function call($seeder, $callDependencies = true)
{
if ($this->seedPath === null)
{
$this->seedPath = APPPATH . 'database/seeds/';
}

$obj = $this->loadSeeder($seeder);
if ($callDependencies === true && $obj instanceof Seeder) {
$obj->callDependencies($this->seedPath);
}
$obj->run();
}

/**
* Get Seeder instance
*
* @param string $seeder
* @return Seeder
*/
protected function loadSeeder($seeder)
{
$file = $this->seedPath . $seeder . '.php';
require_once $file;

return new $seeder;
}

/**
* Call dependency seeders
*
* @param string $seedPath
*/
public function callDependencies($seedPath)
{
foreach ($this->depends as $path => $seeders) {
$this->seedPath = $seedPath;
if (is_string($path)) {
$this->setPath($path);
}

$this->callDependency($seeders);
}
$this->setPath($seedPath);
}

/**
* Call dependency seeder
*
* @param string|array $seederName
*/
protected function callDependency($seederName)
{
if (is_array($seederName)) {
array_map([$this, 'callDependency'], $seederName);
return;
}

$seeder = $this->loadSeeder($seederName);
if (is_string($this->seedPath)) {
$seeder->setPath($this->seedPath);
}

$seeder->call($seederName, true);
}

/**
* Set path for seeder files
*
* @param string $path
*/
public function setPath($path)
{
$this->seedPath = rtrim($path, '/').'/';
}

public function __get($property)
{
return $this->CI->$property;
}
}
8 changes: 8 additions & 0 deletions application/tests/DbTestCase.php
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,12 @@

class DbTestCase extends CIPHPUnitTestDbTestCase
{
public static function setUpBeforeClass(): void
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Normally, my preference is to reset the data between each test rather than between each test class, so that there's no leakage between tests, but it seems like setting up such a large database is quite expensive. (On my humble machine, this takes between 30 to 40s to do the reload.)

I think there's a few options we could explore for speeding this up:

  • Use a reduced-size data set for the tests - Less production-like, but fast and you get some data set up for free
  • Use a schema-only data set for the tests - Even less production like, but you get full control of the data being set up and it should reload really fast
  • Maybe: Benchmark the DatabaseSeeder and see if we can get that to run faster somehow, though it's probably just the data size making it slow

{
parent::setUpBeforeClass();

$CI =& get_instance();
$CI->load->library('Seeder');
$CI->seeder->call('DatabaseSeeder');
}
}
31 changes: 31 additions & 0 deletions application/tests/controllers/API_test.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
<?php

class API_test extends DbTestCase
{

public function test_coverart() {
$output = $this->request(
'GET',
'/api/feed/audiobooks/id/52',
[
'coverart' => 1,
'format' => 'json',
]
);
$books = json_decode($output);
$this->assertEquals(
$books[0]['coverart_jpg'],
'https://www.archive.org/download/LibrivoxCdCoverArt12/Letters_Two_Brides_1110.jpg'
);
$this->assertEquals(
$books[0]['coverart_pdf'],
'https://www.archive.org/download/LibrivoxCdCoverArt12/Letters_Two_Brides_1110.pdf'
);
$this->assertEquals(
$books[0]['coverart_thumbnail'],
'https://www.archive.org/download/LibrivoxCdCoverArt12/Letters_Two_Brides_1110_thumb.jpg'
);
}
}

?>