Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Update and test fastx-tools #375

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
+362 −289
Open

WIP: Update and test fastx-tools #375

Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
a81f7d5
Test and update fastx tools
bgruening May 23, 2016
a554313
Upgrade to newer version of fastx
bgruening May 23, 2016
459ff1c
Add tests for the remaining fastx tools
mvdbeek Sep 17, 2016
8865acd
Explicitly set `-Q` for all non-cs fastq variants
mvdbeek Sep 18, 2016
0c1f99d
Close command tag
mvdbeek Sep 18, 2016
f4e5d27
Fix command tag ... again
mvdbeek Sep 18, 2016
f206706
Fix requirements
mvdbeek Sep 18, 2016
361f286
Don't fail with gnuplot warnings
mvdbeek Sep 18, 2016
0024992
Use simsize comparison for png
mvdbeek Sep 18, 2016
3fdc28d
Retab certain fastx tools, remove duplicate macro imports
mvdbeek Jan 1, 2017
d55db72
Retab more fastx tools
mvdbeek Jan 1, 2017
f9a5168
Replace '.ext ==' pattern with is_of_type
Jan 20, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add tests for the remaining fastx tools
  • Loading branch information
@mvdbeek
mvdbeek committed Jan 1, 2017
commit 459ff1c28f7e2e0e7a7c0d2af18f4f191fd1fd72
26 changes: 17 additions & 9 deletions tool_collections/fastx_toolkit/fasta_clipping_histogram/fasta_clipping_histogram.xml
View file
Original file line number Diff line number Diff line change
@@ -1,20 +1,28 @@
<tool id="cshl_fasta_clipping_histogram" name="Length Distribution" version="1.14">
<tool id="cshl_fasta_clipping_histogram" name="Length Distribution" version="@VERSION@">
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this would cause problems, unless the version lineage code recognizes that 0.0.14 > 1.0.0

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yes, this is true. Too bad. Are you working on this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was planning on moving it to IUC in galaxyproject/tools-iuc#1413, and of course I can merge in your changes there to avoid reinventing effort.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool, please go ahead and close this when you are ready.

<description>chart</description>
<requirements>
<requirement type="package" version="0.0.14">fastx_toolkit</requirement>
</requirements>
<command>
fasta_clipping_histogram.pl
'$input'
'$outfile'
</command>
<expand macro="requirements" />
<macros>
<import>fastx_macros.xml</import>
</macros>
<macros>
<import>fastx_macros.xml</import>
</macros>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The macro is imported twice.

<command><![CDATA[
command -v perl5.22.0 &&
perl5.22.0 `command -v fasta_clipping_histogram.pl` '$input' '$outfile' ||
fasta_clipping_histogram.pl '$input' '$outfile'
]]></command>
<inputs>
<param format="fasta" name="input" type="data" label="Library to analyze" />
</inputs>
<outputs>
<data format="png" name="outfile" metadata_source="input" />
</outputs>
<tests>
<test>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bgruening unfortunately this test only passes if I set galaxy's conda version to 4.2.7 :(.
Otherwise I'm getting fastx_toolkit==0.0.14-2 instead of -3. See https://travis-ci.org/galaxyproject/tools-devteam/jobs/160679791#L355

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Urgs, this offline bug needs to be fixed. This is really killing us.

<param name="input" value="input.fasta" ftype="fasta"/>
<output name="outfile" file="histogram.pdf" />
</test>
</tests>
<help>
**What it does**
1 change: 1 addition & 0 deletions tool_collections/fastx_toolkit/fasta_clipping_histogram/fastx_macros.xml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../fastx_macros.xml
6 changes: 6 additions & 0 deletions tool_collections/fastx_toolkit/fasta_clipping_histogram/test-data/input.fasta
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
>sequence1
AGTAGTAGGTGATGTAGAGAGAGAGAGAGTAG
>sequence2
GTGTGTGTGGGAAGTTGACACAGTA
>sequence3
CCTTGAGATTAACGCTAATCAAGTAAAC
6 changes: 0 additions & 6 deletions tool_collections/fastx_toolkit/fasta_clipping_histogram/tool_dependencies.xml
View file

This file was deleted.

15 changes: 8 additions & 7 deletions tool_collections/fastx_toolkit/fasta_formatter/fasta_formatter.xml
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
<tool id="cshl_fasta_formatter" version="1.14" name="FASTA Width">
<tool id="cshl_fasta_formatter" version="@VERSION@" name="FASTA Width">
<description>formatter</description>
<requirements>
<requirement type="package" version="0.0.14">fastx_toolkit</requirement>
</requirements>
<expand macro="requirements" />
<macros>
<import>fastx_macros.xml</import>
</macros>
<!--
Note:
fasta_formatter also has a tabular output mode (-t),
@@ -13,9 +14,9 @@
FASTA file.
-->
<command>
<![CDATA[
zcat -f < '$input' | fasta_formatter -w $width -o '$output'
]]>
<![CDATA[
zcat -f < '$input' | fasta_formatter -w $width -o '$output'
]]>
</command>
<inputs>
<param format="fasta" name="input" type="data" label="Library to re-format" />
1 change: 1 addition & 0 deletions tool_collections/fastx_toolkit/fasta_formatter/fastx_macros.xml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../fastx_macros.xml
6 changes: 0 additions & 6 deletions tool_collections/fastx_toolkit/fasta_formatter/tool_dependencies.xml
View file

This file was deleted.

9 changes: 5 additions & 4 deletions tool_collections/fastx_toolkit/fasta_nucleotide_changer/fasta_nucleotide_changer.xml
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
<tool id="cshl_fasta_nucleotides_changer" version="1.0.0" name="RNA/DNA" >
<tool id="cshl_fasta_nucleotides_changer" version="@VERSION@" name="RNA/DNA" >
<description>converter</description>
<requirements>
<requirement type="package" version="0.0.13">fastx_toolkit</requirement>
</requirements>
<expand macro="requirements" />
<macros>
<import>fastx_macros.xml</import>
</macros>
<command>
<![CDATA[
zcat -f < '$input' | fasta_nucleotide_changer -$mode -v -o '$output'
1 change: 1 addition & 0 deletions tool_collections/fastx_toolkit/fasta_nucleotide_changer/fastx_macros.xml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../fastx_macros.xml
20 changes: 14 additions & 6 deletions tool_collections/fastx_toolkit/fastq_quality_boxplot/fastq_quality_boxplot.xml
View file
Original file line number Diff line number Diff line change
@@ -1,17 +1,25 @@
<tool id="cshl_fastq_quality_boxplot" name="Draw quality score boxplot" version="1.0.0">
<tool id="cshl_fastq_quality_boxplot" name="Draw quality score boxplot" version="@VERSION@">
<description></description>
<requirements>
<requirement type="package" version="0.0.13">fastx_toolkit</requirement>
</requirements>
<command>fastq_quality_boxplot_graph.sh -t '$input.name' -i $input -o $output</command>

<expand macro="requirements" />
<macros>
<import>fastx_macros.xml</import>
</macros>
<command>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indentation

fastq_quality_boxplot_graph.sh -t '$input.name' -i '$input' -o '$output'
</command>
<inputs>
<param format="txt" name="input" type="data" label="Statistics report file" help="output of 'FASTQ Statistics' tool" />
</inputs>

<outputs>
<data format="png" name="output" metadata_source="input" />
</outputs>
<tests>
<test>
<param name="input" value="fastq_stats1.out" ftype="txt"></param>
<output name="output" file="quality_boxplot_out.png"></output>
</test>
</tests>
<help>

**What it does**
1 change: 1 addition & 0 deletions tool_collections/fastx_toolkit/fastq_quality_boxplot/fastx_macros.xml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../fastx_macros.xml
37 changes: 37 additions & 0 deletions tool_collections/fastx_toolkit/fastq_quality_boxplot/test-data/fastq_stats1.out
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
column count min max sum mean Q1 med Q3 IQR lW rW A_Count C_Count G_Count T_Count N_Count Max_count
1 9 23 34 288 32.00 33 33 33 0 33 33 3 1 4 1 0 9
2 9 28 33 287 31.89 31 33 33 2 28 33 3 3 2 1 0 9
3 9 13 34 268 29.78 28 33 33 5 21 34 5 1 0 3 0 9
4 9 17 33 261 29.00 30 33 33 3 26 33 1 2 3 3 0 9
5 9 22 33 269 29.89 30 33 33 3 26 33 3 3 3 0 0 9
6 9 22 33 277 30.78 30 33 33 3 26 33 5 3 0 1 0 9
7 9 21 33 258 28.67 24 33 33 9 21 33 4 1 3 1 0 9
8 9 12 33 263 29.22 32 33 33 1 31 33 2 1 1 5 0 9
9 9 29 33 290 32.22 33 33 33 0 33 33 3 3 2 1 0 9
10 9 23 33 277 30.78 32 33 33 1 31 33 1 4 2 2 0 9
11 9 12 33 245 27.22 21 31 33 12 12 33 5 2 1 1 0 9
12 9 13 33 214 23.78 15 24 33 18 13 33 2 4 2 1 0 9
13 9 5 33 249 27.67 29 31 33 4 23 33 2 1 1 5 0 9
14 9 5 33 233 25.89 24 33 33 9 11 33 3 3 2 1 0 9
15 9 15 33 251 27.89 24 33 33 9 15 33 5 1 1 2 0 9
16 9 23 34 269 29.89 24 33 33 9 23 34 3 1 2 3 0 9
17 9 13 34 266 29.56 33 33 33 0 33 33 2 3 1 3 0 9
18 9 21 34 272 30.22 31 33 33 2 28 34 0 5 1 3 0 9
19 9 5 34 244 27.11 27 30 33 6 18 34 4 4 1 0 0 9
20 9 11 34 241 26.78 23 32 33 10 11 34 3 4 2 0 0 9
21 9 13 33 240 26.67 24 27 33 9 13 33 1 4 0 4 0 9
22 9 5 33 190 21.11 13 21 33 20 5 33 1 4 0 3 1 9
23 9 5 33 205 22.78 16 26 33 17 5 33 4 4 1 0 0 9
24 9 5 33 247 27.44 28 31 33 5 21 33 1 5 1 2 0 9
25 9 11 34 241 26.78 24 33 33 9 11 34 3 4 0 2 0 9
26 9 5 33 212 23.56 18 31 33 15 5 33 0 6 0 3 0 9
27 9 5 33 227 25.22 21 26 33 12 5 33 3 4 1 1 0 9
28 9 21 33 255 28.33 24 31 33 9 21 33 2 4 3 0 0 9
29 9 5 33 228 25.33 21 30 33 12 5 33 2 4 1 2 0 9
30 9 10 33 213 23.67 16 28 33 17 10 33 3 4 2 0 0 9
31 9 5 33 236 26.22 21 31 33 12 5 33 1 4 1 3 0 9
32 9 5 33 210 23.33 12 29 33 21 5 33 3 3 0 3 0 9
33 9 5 33 183 20.33 9 21 33 24 5 33 1 4 2 2 0 9
34 9 5 33 150 16.67 7 17 22 15 5 33 3 4 1 1 0 9
35 9 13 33 217 24.11 21 24 29 8 13 33 1 4 1 3 0 9
36 9 5 33 195 21.67 18 21 32 14 5 33 3 2 1 3 0 9
Binary file added ...llections/fastx_toolkit/fastq_quality_boxplot/test-data/quality_boxplot_out.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
6 changes: 0 additions & 6 deletions tool_collections/fastx_toolkit/fastq_quality_boxplot/tool_dependencies.xml
View file

This file was deleted.

9 changes: 5 additions & 4 deletions tool_collections/fastx_toolkit/fastq_quality_converter/fastq_quality_converter.xml
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
<tool id="cshl_fastq_quality_converter" name="Quality format converter" version="1.0.0">
<tool id="cshl_fastq_quality_converter" name="Quality format converter" version="@VERSION@">
<description>(ASCII-Numeric)</description>
<requirements>
<requirement type="package" version="0.0.13">fastx_toolkit</requirement>
</requirements>
<expand macro="requirements" />
<macros>
<import>fastx_macros.xml</import>
</macros>
<command>
<![CDATA[
zcat -f < '$input' | fastq_quality_converter $QUAL_FORMAT -o '$output' -Q $offset
1 change: 1 addition & 0 deletions tool_collections/fastx_toolkit/fastq_quality_converter/fastx_macros.xml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../fastx_macros.xml
6 changes: 0 additions & 6 deletions tool_collections/fastx_toolkit/fastq_quality_converter/tool_dependencies.xml
View file

This file was deleted.

9 changes: 5 additions & 4 deletions tool_collections/fastx_toolkit/fastq_quality_filter/fastq_quality_filter.xml
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
<tool id="cshl_fastq_quality_filter" version="1.0.0" name="Filter by quality">
<tool id="cshl_fastq_quality_filter" version="@VERSION@" name="Filter by quality">
<description></description>
<requirements>
<requirement type="package" version="0.0.13">fastx_toolkit</requirement>
</requirements>
<expand macro="requirements" />
<macros>
<import>fastx_macros.xml</import>
</macros>
<command>
<![CDATA[
zcat -f < '$input' | fastq_quality_filter -q $quality -p $percent -v -o '$output'
1 change: 1 addition & 0 deletions tool_collections/fastx_toolkit/fastq_quality_filter/fastx_macros.xml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../fastx_macros.xml
6 changes: 0 additions & 6 deletions tool_collections/fastx_toolkit/fastq_quality_filter/tool_dependencies.xml
View file

This file was deleted.

15 changes: 8 additions & 7 deletions tool_collections/fastx_toolkit/fastq_to_fasta/fastq_to_fasta.xml
View file
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
<tool id="cshl_fastq_to_fasta" name="FASTQ to FASTA" version="1.0.0">
<tool id="cshl_fastq_to_fasta" name="FASTQ to FASTA" version="@VERSION@">
<description>converter from FASTX-toolkit</description>
<requirements>
<requirement type="package" version="0.0.13">fastx_toolkit</requirement>
</requirements>
<expand macro="requirements" />
<macros>
<import>fastx_macros.xml</import>
</macros>
<command>gunzip -cf $input | fastq_to_fasta $SKIPN $RENAMESEQ -o $output -v
#if $input.ext == "fastqsanger":
-Q 33
#end if
#if $input.ext == "fastqsanger":
-Q 33
#end if
</command>

<inputs>
1 change: 1 addition & 0 deletions tool_collections/fastx_toolkit/fastq_to_fasta/fastx_macros.xml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../fastx_macros.xml
6 changes: 0 additions & 6 deletions tool_collections/fastx_toolkit/fastq_to_fasta/tool_dependencies.xml
View file

This file was deleted.

9 changes: 5 additions & 4 deletions tool_collections/fastx_toolkit/fastx_artifacts_filter/fastx_artifacts_filter.xml
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
<tool id="cshl_fastx_artifacts_filter" version="1.0.0" name="Remove sequencing artifacts">
<tool id="cshl_fastx_artifacts_filter" version="@VERSION@" name="Remove sequencing artifacts">
<description></description>
<requirements>
<requirement type="package" version="0.0.13">fastx_toolkit</requirement>
</requirements>
<expand macro="requirements" />
<macros>
<import>fastx_macros.xml</import>
</macros>
<command>
<![CDATA[
zcat -f < '$input' | fastx_artifacts_filter -v -o '$output'
1 change: 1 addition & 0 deletions tool_collections/fastx_toolkit/fastx_artifacts_filter/fastx_macros.xml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../fastx_macros.xml
6 changes: 0 additions & 6 deletions tool_collections/fastx_toolkit/fastx_artifacts_filter/tool_dependencies.xml
View file

This file was deleted.

12 changes: 7 additions & 5 deletions tool_collections/fastx_toolkit/fastx_barcode_splitter/fastx_barcode_splitter.xml
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
<tool id="cshl_fastx_barcode_splitter" version="1.0.0" name="Barcode Splitter">
<tool id="cshl_fastx_barcode_splitter" version="@VERSION@" name="Barcode Splitter">
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lparsons would you mind if I pull in your fastx_barcode_splitter_enhanced tool as a replacement? I think your version is much more useful.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mvdbeek Not at all. How can I help with it?

Copy link
Member

@mvdbeek mvdbeek Sep 19, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR has been opened from the test_fastx branch, if you copy over the tool files here and do a PR selecting the test_fastx brach, we can get your version into this PR.

<description></description>
<requirements>
<requirement type="package" version="0.0.13">fastx_toolkit</requirement>
</requirements>
<command interpreter="bash">fastx_barcode_splitter_galaxy_wrapper.sh '$BARCODE' '$input' "$input.name" "$output.files_path" --mismatches $mismatches --partial $partial $EOL > '$output' </command>
<expand macro="requirements" />
<macros>
<import>fastx_macros.xml</import>
</macros>
<command>
'$__tool_directory__'/fastx_barcode_splitter_galaxy_wrapper.sh '$BARCODE' '$input' "$input.name" "$output.files_path" --mismatches $mismatches --partial $partial $EOL > '$output' </command>

<inputs>
<param format="txt" name="BARCODE" type="data" label="Barcodes to use" />
1 change: 1 addition & 0 deletions tool_collections/fastx_toolkit/fastx_barcode_splitter/fastx_macros.xml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../fastx_macros.xml
6 changes: 0 additions & 6 deletions tool_collections/fastx_toolkit/fastx_barcode_splitter/tool_dependencies.xml
View file

This file was deleted.

9 changes: 5 additions & 4 deletions tool_collections/fastx_toolkit/fastx_clipper/fastx_clipper.xml
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
<tool id="cshl_fastx_clipper" name="Clip" version="1.0.1">
<tool id="cshl_fastx_clipper" name="Clip" version="@VERSOIN@">
<description>adapter sequences</description>
<requirements>
<requirement type="package" version="0.0.13">fastx_toolkit</requirement>
</requirements>
<expand macro="requirements" />
<macros>
<import>fastx_macros.xml</import>
</macros>
<command>
<![CDATA[
zcat -f < '$input' | fastx_clipper -l $minlength -a '$clip_source.clip_sequence' -d $keepdelta -o '$output' -v $KEEP_N $DISCARD_OPTIONS
1 change: 1 addition & 0 deletions tool_collections/fastx_toolkit/fastx_clipper/fastx_macros.xml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../fastx_macros.xml
6 changes: 0 additions & 6 deletions tool_collections/fastx_toolkit/fastx_clipper/tool_dependencies.xml
View file

This file was deleted.

21 changes: 11 additions & 10 deletions tool_collections/fastx_toolkit/fastx_collapser/fastx_collapser.xml
View file
Original file line number Diff line number Diff line change
@@ -1,15 +1,16 @@
<tool id="cshl_fastx_collapser" version="1.0.0" name="Collapse">
<tool id="cshl_fastx_collapser" version="@VERSION@" name="Collapse">
<description>sequences</description>
<requirements>
<requirement type="package" version="0.0.13">fastx_toolkit</requirement>
</requirements>
<expand macro="requirements" />
<macros>
<import>fastx_macros.xml</import>
</macros>
<command>
<![CDATA[
zcat -f < '$input' | fastx_collapser -v -o '$output'
#if $input.ext == "fastqsanger":
-Q 33
#end if
]]>
<![CDATA[
zcat -f < '$input' | fastx_collapser -v -o '$output'
#if $input.ext == "fastqsanger":
-Q 33
#end if
]]>
</command>

<inputs>
1 change: 1 addition & 0 deletions tool_collections/fastx_toolkit/fastx_collapser/fastx_macros.xml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../fastx_macros.xml
6 changes: 0 additions & 6 deletions tool_collections/fastx_toolkit/fastx_collapser/tool_dependencies.xml
View file

This file was deleted.

10 changes: 10 additions & 0 deletions tool_collections/fastx_toolkit/fastx_macros.xml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
<macros>
<token name="@VERSION@">1.14</token>

<xml name="requirements">
<requirements>
<requirement type="package" version="0.0.14">fastx_toolkit</requirement>
<yield />
</requirements>
</xml>
</macros>
1 change: 1 addition & 0 deletions tool_collections/fastx_toolkit/fastx_nucleotides_distribution/fastx_macros.xml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../fastx_macros.xml
Loading