Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update for pg 0.11.0 and a few minor changes #28

Open
wants to merge 85 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
0390d78
treat mysql float as pg real
jcdny Oct 27, 2010
a49cf08
dont guess at precision
jcdny Oct 27, 2010
61bcbbf
check number of tuples inserted on copy
jcdny Oct 27, 2010
9dd8a2f
fix error on counting inserts for empty tables
jcdny Oct 28, 2010
2bfa98f
clean up formatting
tardate Nov 1, 2010
6a4a8b7
factor out set serial sequence code to postgres_writer
tardate Nov 1, 2010
d8c672e
merge from main repo
jcdny Nov 4, 2010
9727863
remerge of bug fixes
jcdny Nov 4, 2010
7b48c1c
allow default of no socket to be entered
tardate Nov 20, 2010
fc54c74
ignore redcar, rvmrc configurations
tardate Nov 20, 2010
e01f5df
add basic auto-increment test (for file coversions)
tardate Nov 20, 2010
8d8883b
fix indenting
tardate Nov 20, 2010
c4b20c2
floats and reals should migrate to double precision (although floats …
tardate Nov 20, 2010
1d44286
readme format fixup
tardate Nov 20, 2010
43c8930
merge jcdny/master and ensure float tests pass
tardate Nov 20, 2010
ee7cb8e
revised/corrected integer signed/unsigned conversions
tardate Nov 20, 2010
4e6f1e5
simplify tinyint type matching
tardate Nov 20, 2010
2b7664a
testing this with pg 0.10.+
Apr 9, 2011
a1d1755
remove deprecated feature per gem -v 1.7.2
Apr 9, 2011
cada55b
add a check for mysql cmd, on my mac it's mysql5
Apr 11, 2011
12f0b43
resync gemspec and rakefile
tardate Apr 30, 2011
2bae286
update pg version to ~> 0.11.0
May 2, 2011
3cce625
resync rakefile + gemspec for pg ~> 0.11.0
tardate May 2, 2011
a554cae
add CHANGELOG and step gem version to 0.1.1
tardate May 2, 2011
dbef107
Update README to reflect gem availability
tardate May 2, 2011
8c6a1fe
make tests pass by moving startup/shutdown into setup/teardown. fixes…
Aug 26, 2011
3db085b
Fix indentation and remove a tab.
jcoleman Sep 7, 2011
8e624ae
Remove dreadful nested ternary.
jcoleman Sep 7, 2011
a1fdd91
Fix boolean parsing bug.
jcoleman Sep 7, 2011
10f92f5
make tests pass by moving startup/shutdown into setup/teardown. fixes…
Aug 26, 2011
0ab0487
Merge branch 'master' of github.com:jcoleman/mysql2postgres into bool…
jcoleman Sep 8, 2011
1e1ed6a
Remove unused variable.
jcoleman Sep 8, 2011
32da4e0
Add assert_nil extension to Test::Unit.
jcoleman Sep 8, 2011
7432927
Fix boolean handling of BIT(1) type columns.
jcoleman Sep 8, 2011
f032930
Remove incorrectly indented space.
jcoleman Sep 8, 2011
7f5d030
Don't do unnecessary calculation.
jcoleman Sep 8, 2011
98d40dd
Factor column information methods.
jcoleman Sep 8, 2011
f9b6aca
Note about boolean issues is no longer applicable.
jcoleman Sep 8, 2011
ad4a92d
Don't copy views as if they were tables.
jcoleman Sep 9, 2011
7cac276
Cleanup the row processing logic.
jcoleman Sep 9, 2011
ad32f85
Cleanup date* types column default processing.
jcoleman Sep 9, 2011
235f157
Fix fixnum to string casting bug.
jcoleman Sep 9, 2011
135853d
Merge pull request #3 from apeckham/master
tardate Sep 11, 2011
cb4a549
Merge pull request #4 from jcoleman/boolean_parsing_bug
tardate Sep 11, 2011
99dbd98
Merge pull request #6 from jcoleman/factor_column_information
tardate Sep 11, 2011
515129e
Merge pull request #7 from jcoleman/master
tardate Sep 12, 2011
d5acb4f
add two missing tests for null conversion and datetime conversion
Sep 14, 2011
49eac20
add test coverage for unique indexes, and indexes that have to be ren…
Sep 14, 2011
a80fa1f
add test coverage for foreign keys
Sep 14, 2011
1640a15
fix rcov rake task
Sep 14, 2011
926aa77
remove unused method which was accidentally defined twice
Sep 14, 2011
d6d2bcb
add missing test for datetime defaults, increase test coverage to 88%
Sep 14, 2011
690b23b
add test for time conversion
Sep 14, 2011
960041f
add test for what the tool prints to stdout
Sep 14, 2011
2637289
add test for force_truncate option, and fix bug where when option was…
Sep 14, 2011
af016ae
add missing test for enum conversion
Sep 14, 2011
1335bab
Merge https://github.com/apeckham/mysql2postgres
wranai Sep 16, 2011
54ffe74
quoting default values in table definition
wranai Sep 16, 2011
5005daa
Merge https://github.com/maxlapshin/mysql2postgres
wranai Sep 16, 2011
4e3feac
Refactor string escaping.
jcoleman Sep 19, 2011
89b3a33
Merge branch 'master' of github.com:apeckham/mysql2postgres
jcoleman Sep 19, 2011
f284ad2
Option to create columns with or without timezones.
jcoleman Sep 19, 2011
561dec9
correct projectwide spelling of 'suppress'
jtippett Oct 11, 2011
1d10e57
add support for suppressing indexes
jtippett Oct 11, 2011
ab23963
typo
jtippett Oct 11, 2011
35c5c88
properly escape varchar default value so pgsql does not confuse it wi…
jtippett Oct 11, 2011
ef9951e
Allow Postgres to rename indexes; don't try to preserve MySQL index n…
Oct 13, 2011
d3bee02
Add capability to update sequence values without writing DDL.
jcoleman Nov 2, 2011
f3d5cfa
bumped pg gem to 0.11.0
Nov 24, 2011
8477333
Fix homepage link
mnaberez Jan 10, 2012
c3272b8
Don't fail on nil date/time.
seven1m Jan 11, 2012
bbd31d7
Default to 5 digit precision.
seven1m Jan 11, 2012
db137fa
Use real for decimal
seven1m Jan 11, 2012
136ad18
Merge pull request #9 from jcoleman/master
tardate Feb 11, 2012
727d9cd
Merge pull request #8 from apeckham/master
tardate Feb 11, 2012
e2d61b3
fix test_truncate(ConvertToFileTest) failure
tardate Feb 12, 2012
3d569a5
fix test_index_conversion(ConvertToDbTest) failure
tardate Feb 12, 2012
51d8e6a
Merge remote branch 'jtippett/master' into master
tardate Feb 12, 2012
533032d
clean up after 'jtippett/master' merge
tardate Feb 12, 2012
45739b5
step gem version to 0.2.0
tardate Feb 12, 2012
c89efa5
Merge remote branch 'seven1m/master' into master
tardate Feb 12, 2012
9494277
Merge remote branch 'dakhota/master' into master
tardate Feb 12, 2012
62d0180
update gemspec
tardate Feb 12, 2012
bf67dd1
Merge remote branch 'mattsoldo/master' into master
tardate Feb 12, 2012
b97fad1
Restore conflicting index name migration
tardate Feb 12, 2012
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,8 @@
configs
test/fixtures/test*.sql
pkg
.redcar
.rvmrc
.bundle
*.out
rdoc
19 changes: 19 additions & 0 deletions CHANGELOG
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
0.2.2 Feb 13, 2012
===

* merge contributions from: James Coleman, Aaron Peckham, James Tippett
* breaking change: note that "supress*" configuration parameters now use correct spelling. Existing config files you may have will need housekeeping.
* new configuration options: suppress_sequence_update; suppress_indexes; use_timezones
* adds a Gemfile to help setup development environment

0.1.1 May 3, 2011
===

* updated for pg driver ~> 0.11.0
* add a check for mysql cmd, e.g. mysql or mysql5
* remove deprecated feature per gem -v 1.7.2

0.1.0 September 18, 2010
===

* Initial packaging as a gem
10 changes: 6 additions & 4 deletions Gemfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
source :rubygems

gem 'mysql'
gem 'pg'
gem 'test-unit'
gem 'jeweler'
group :development do
gem "bundler", "~> 1.0.21"
gem "jeweler", "~> 1.5.2"
end

gemspec
27 changes: 20 additions & 7 deletions Gemfile.lock
Original file line number Diff line number Diff line change
@@ -1,21 +1,34 @@
PATH
remote: .
specs:
mysql2psql (0.2.0)
mysql (= 2.8.1)
mysql2psql
pg (~> 0.11.0)

GEM
remote: http://rubygems.org/
specs:
git (1.2.5)
jeweler (1.6.0)
jeweler (1.5.2)
bundler (~> 1.0.0)
git (>= 1.2.5)
rake
json (1.6.5)
mysql (2.8.1)
pg (0.11.0)
rake (0.8.7)
test-unit (2.3.0)
rake (0.9.2.2)
rdoc (3.12)
json (~> 1.4)
test-unit (2.4.7)

PLATFORMS
ruby

DEPENDENCIES
jeweler
mysql
pg
test-unit
bundler (~> 1.0.21)
jeweler (~> 1.5.2)
mysql2psql!
rake (~> 0.9.2.2)
rdoc (~> 3.12)
test-unit (>= 2.1.1)
101 changes: 56 additions & 45 deletions README.rdoc
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,8 @@ to contact me, I'll help you.
Mysql2psql is packaged as a gem. Install as usual (use sudo if required).

$ gem install mysql2psql
NB: the gem hasn't been formally released on http://rubygems.org/ yet. For now you need to clone the git repo at http://github.com/tardate/mysql2postgres and 'rake install'

After installation, the "mysql2psql" command will be available.
After installation, the "mysql2psql" command will be available.
When run, it will generate a default configuration file in the current directory if a config
file is not already found. The default configuration filename is mysql2psql.yml.

Expand Down Expand Up @@ -42,83 +41,93 @@ After editing the configuration file and launching tool, you will see something


== Database driver dependencies
Mysql2psql uses the 'mysql' and 'pg' gems for database connectivity.
Mysql2psql uses the 'mysql' and 'pg' gems for database connectivity.
Mysql2psql gem installation should automatically install these too.

If you encounter any issues with db connectivity, verify that the 'mysql' and 'pg' gems are installed and working correctly first.
The 'mysql' gem in particular may need a hint on where to find the mysql headers and libraries:

$ [sudo] gem install mysql -- --with-mysql-dir=/usr/local/mysql

NB: With Ruby 1.8.x, the gem install will usually complain about "No definition for ..." errors. These are non-fatal and just affect the documentation install. This doesn't happen with Ruby 1.9.2.

With bundler you can do a simple: bundle install

NB: With Ruby 1.8.x, the gem install will usually complain about "No definition for ..." errors. These are non-fatal and just affect the documentation install. This doesn't happen with Ruby 1.9.2.

== Contributing to Mysql2psql

* Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet
* Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it
* Fork the project
* Start a feature/bugfix branch
* Commit and push until you are happy with your contribution
* Make sure to add tests for it. This is important so no-one unintentionally breaks it in a future version.
* Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so the gem maintainers can cherry-pick around it.
* Send a pull request

== Getting the source
Mysql2psql was first produced by Max Lapshin <[email protected]> who maintains the master
repository at http://github.com/maxlapshin/mysql2postgres
Mysql2psql was first produced by Max Lapshin <[email protected]> who maintains the master
repository at https://github.com/maxlapshin/mysql2postgres

The gem packaged version you are currently looking at has yet to be merged into the master repo.
The gem bundling and refactoring for testing was done by Paul Gallagher <[email protected]> and
the repository is at http://github.com/tardate/mysql2postgres
the repository is at https://github.com/tardate/mysql2postgres

Gem packaging is done with Jeweler. Note that on windows this means you must run under mingw or other shell that supports git to do the gem bundling (but you can gem install the built gem in the normal Windows shell).

== Setting up for development

The project includes a Gemfile so with bundler you can install gem dependencies with:

gem install bundler # if not already installed
bundle install

Note comments 'Database driver dependencies' regarding issues that you may encounter with the mysql gem.

== Running tests
If you fork/clone the project, you will want to run the test suite (and add to it if you are developing new features or fixing bugs).

A suite of tests are provided and are run using rake (rake -T for more info)
rake test:units
rake test:integration
rake test
rake test

Rake without parameters (or "rake test") will run both the unit and integration tests.

Unit tests are small standalone tests of the mysql2psql codebase that have no external dependencies (like a database)

Integration tests require suitable mysql and postgres databases to be setup in advance.
Integration tests require suitable mysql and postgres databases to be setup in advance.
Running the integration tests *will* rewrite data and schema for the "mysql2psql_test" databases in mysql and postgres. Don't use this database for anything else!

mysql on localhost:3306
- database created called "mysql2psql_test"
- user setup for "mysql2psql" with no password
- e.g.
Setting up mysql on localhost:3306
- create a database called "mysql2psql_test"
- setup a user "mysql2psql" with no password

e.g.

mysqladmin -uroot -p create mysql2psql_test
mysql -uroot -p -e "grant all on mysql2psql_test.* to 'mysql2psql'@'localhost';"
# verify connecction:
# verify connection:
mysql -umysql2psql -e "select database(),'yes' as connected" mysql2psql_test

postgres on localhost:5432
- database created called "mysql2psql_test"
- role (user) access setup for "mysql2psql" with no password
- e.g.


Setting up PostgreSQL on localhost:5432
- create a database called "mysql2psql_test"
- setup a role (user) "mysql2psql" with no password

e.g.

psql postgres -c "create role mysql2psql with login"
psql postgres -c "create database mysql2psql_test with owner mysql2psql encoding='UTF8'"
# verify connection:
psql mysql2psql_test -U mysql2psql -c "\c"

== Notes, Limitations, Outstanding Issues..

Todo:
- more tests
- release gem
- a windows cmd shim
== Notes, Limitations, Outstanding Issues..

=== note from mgkimsal
I'm still having trouble with bit(1)/boolean fields
workaround I've found is to put output in file
then in VIM on file, search/replace the true/false binary fields with t/f
specifically
* more test coverage, as always...

(reversed on 3/23 - should be ^A gets f)
:%s/^@/t/g
:%s/^A/f/g
keystrokes are ctrl-v ctrl-shift-@ to get the 'true' binary field
keystrokes are ctrl-v ctrl-shift-A to get the 'false' binary field

== Contributors
Project founded by Max Lapshin <[email protected]>

Other contributors (in git log order):
Contributors (roughly git log order):
- Anton Ageev <[email protected]>
- Samuel Tribehou <[email protected]>
- Marco Nenciarini <[email protected]>
Expand All @@ -130,8 +139,10 @@ Other contributors (in git log order):
- Jacob Coby <[email protected]>
- Neszt Tibor <[email protected]>
- Miroslav Kratochvil <[email protected]>
- Paul Gallagher <[email protected]>




- {Paul Gallagher}[https://github.com/tardate] <[email protected]>
- {James Coleman}[https://github.com/jcoleman] <[email protected]>
- {Aaron Peckham}[https://github.com/apeckham]
- {James Tippett}[https://github.com/jtippett]
- {Tim Morgan}[https://github.com/seven1m]
- {dakhota}[https://github.com/dakhota]
- {Matthew Soldo}[https://github.com/mattsoldo]
28 changes: 18 additions & 10 deletions Rakefile
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ begin
gem.description = %Q{It can create postgresql dump from mysql database or directly load data from mysql to
postgresql (at about 100 000 records per minute). Translates most data types and indexes.}
gem.email = "[email protected]"
gem.homepage = "http://github.com/tardate/mysql2postgresql"
gem.homepage = "https://github.com/tardate/mysql2postgres"
gem.authors = [
"Max Lapshin <[email protected]>",
"Anton Ageev <[email protected]>",
Expand All @@ -27,11 +27,19 @@ begin
"Jacob Coby <[email protected]>",
"Neszt Tibor <[email protected]>",
"Miroslav Kratochvil <[email protected]>",
"Paul Gallagher <[email protected]>"
]
"Paul Gallagher <[email protected]>",
"James Coleman <[email protected]>",
"Aaron Peckham",
"James Tippett",
"Tim Morgan",
"dakhota",
"Matthew Soldo"
]
gem.add_dependency "mysql", "= 2.8.1"
gem.add_dependency "pg", "= 0.9.0"
gem.add_dependency "pg", "~> 0.11.0"
gem.add_development_dependency "test-unit", ">= 2.1.1"
gem.add_development_dependency "rake", "~> 0.9.2.2"
gem.add_development_dependency "rdoc", "~> 3.12"
# gem is a Gem::Specification... see http://www.rubygems.org/read/chapter/20 for additional settings
end
Jeweler::GemcutterTasks.new
Expand Down Expand Up @@ -63,25 +71,25 @@ end
begin
require 'rcov/rcovtask'
Rcov::RcovTask.new do |test|
test.libs << 'test'
test.libs << 'lib'
test.libs << 'test/lib'
test.pattern = 'test/**/*test.rb'
test.verbose = true
test.rcov_opts << "--exclude gems/*"
end
rescue LoadError
task :rcov do
abort "RCov is not available. In order to run rcov, you must: sudo gem install spicycode-rcov"
end
end


task :default => :test

require 'rake/rdoctask'
Rake::RDocTask.new do |rdoc|
require 'rdoc/task'
RDoc::Task.new do |rdoc|
version = Mysql2psql::Version::STRING

rdoc.rdoc_dir = 'rdoc'
rdoc.title = "mysql2psql #{version}"
rdoc.rdoc_files.include('README*')
rdoc.rdoc_files.include('lib/**/*.rb')
rdoc.rdoc_files.include("README*", "lib/**/*.rb")
end
Loading