docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock -v $(pwd):$(pwd) -w $(pwd) nextflow/nextflow:24.10.3 bash -c 'nextflow run -r dev nf-core/circrna -params-file params.yaml -profile docker --genome GRCh38' N E X T F L O W ~ version 24.10.3 Pulling nf-core/circrna ... downloaded from https://github.com/nf-core/circrna.git Launching `https://github.com/nf-core/circrna` [admiring_meucci] DSL2 - revision: 925eb02043 [dev] Downloading plugin nf-schema@2.1.1 ------------------------------------------------------ ,--./,-. ___ __ __ __ ___ /,-._.--~' |\ | |__ __ / ` / \ |__) |__ } { | \| | \__, \__/ | \ |___ \`-._,-`-, `._,._,' nf-core/circrna 0.0.1dev ------------------------------------------------------ Input/output options input : samplesheet.csv outdir : results email : leendertcloete@gmail.com multiqc_title : test_circRNA_nfcore circRNA Options tools : ciriquant,circexplorer2,find_circ,circrna_finder,mapsplice,dcc bsj_reads : 5 min_tools : 3 Reference genome options genome : GRCh38 fasta : s3://ngi-igenomes/igenomes//Homo_sapiens/NCBI/GRCh38/Sequence/WholeGenomeFasta/genome.fa gtf : s3://ngi-igenomes/igenomes//Homo_sapiens/NCBI/GRCh38/Annotation/Genes/genes.gtf Generic options trace_report_suffix: 2025-01-20_13-32-43 Core Nextflow options revision : dev runName : admiring_meucci containerEngine : docker launchDir : /home/cloetel/workspace/circrna workDir : /home/cloetel/workspace/circrna/work projectDir : /.nextflow/assets/nf-core/circrna userName : root profile : docker configFiles : !! Only displaying parameters that differ from the pipeline defaults !! ------------------------------------------------------ * The nf-core framework https://doi.org/10.1038/s41587-020-0439-x * Software dependencies https://github.com/nf-core/circrna/blob/master/CITATIONS.md Downloading plugin nf-amazon@2.9.2 executor > local (9) executor > local (10) executor > local (11) executor > local (12) executor > local (13) executor > local (13) executor > local (14) executor > local (14) executor > local (15) executor > local (15) executor > local (16) executor > local (17) executor > local (23) executor > local (24) executor > local (24) executor > local (25) executor > local (25) [54/5d75e5] process > NFCORE_CIRCRNA:CIRCRNA:CAT_FASTQ (fust1_3) [100%] 6 of 6 ✔ [95/af2254] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:CLEAN_FASTA (fasta) [100%] 1 of 1 ✔ [86/e84479] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:GTFFILTER (gtf) [ 0%] 0 of 1 [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:UCSC_GTFTOGENEPRED - [79/5e62df] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:SEQKIT_SPLIT (fasta) [100%] 1 of 1 ✔ [17/cb8171] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:BOWTIE_BUILD (fasta.fa) [ 0%] 0 of 1 [1b/6b051c] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:BOWTIE2_BUILD (fasta.fa) [ 0%] 0 of 1 [bd/0bd0fd] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:BWA_INDEX (fasta.fa) [ 0%] 0 of 1 [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:HISAT2_EXTRACTSPLICESITES - [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:HISAT2_BUILD - [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:STAR_GENOMEGENERATE - [c9/d6fa7a] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:SAMTOOLS_FAIDX (fasta.fa) [100%] 1 of 1 ✔ [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:CIRCEXPLORER2_REFERENCE - [fe/a34870] process > NFCORE_CIRCRNA:CIRCRNA:FASTQC_TRIMGALORE:FASTQC (fust1_3) [100%] 6 of 6 ✔ [f8/721969] process > NFCORE_CIRCRNA:CIRCRNA:FASTQC_TRIMGALORE:TRIMGALORE (fust1_3) [100%] 6 of 6 ✔ [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:STAR2PASS:PASS_1 - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:STAR2PASS:SJDB - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:STAR2PASS:PASS_2 - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCEXPLORER2:PARSE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCEXPLORER2:ANNOTATE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCEXPLORER2:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCRNA_FINDER:MAIN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCRNA_FINDER:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:ALIGN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:SAMTOOLS_INDEX - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:SAMTOOLS_VIEW - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:ANCHORS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:MAIN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRIQUANT:MAIN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRIQUANT:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE1_1ST_PASS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE1_SJDB - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE1_2ND_PASS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE2_1ST_PASS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE2_SJDB - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE2_2ND_PASS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MAIN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:MAPSPLICE:ALIGN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:MAPSPLICE:PARSE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:MAPSPLICE:ANNOTATE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:MAPSPLICE:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:EXTRACT_COUNTS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:COMBINE_COUNTS_PER_TOOL - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FILTER_BSJS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:BED_ADD_SAMPLE_TOOL - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:COMBINE_TOOLS_PER_SAMPLE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:COMBINE_SAMPLES - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:INVESTIGATE_SHIFTS - Plus 45 more processes waiting for tasks… Staging foreign file: s3://ngi-igenomes/igenomes/Homo_sapiens/NCBI/GRCh38/Annotation/Genes/genes.gtf ERROR ~ Error executing process > 'NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:GTFFILTER (gtf)' Caused by: Process `NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:GTFFILTER (gtf)` terminated with an error exit status (1) Command executed [/.nextflow/assets/nf-core/circrna/./workflows/circrna/../../subworkflows/local/../../modules/nf-core/custom/gtffilter/templates/gtffilter.py]: #!/usr/bin/env python # Written by Olga Botvinnik with subsequent reworking by Jonathan Manning and Nico Trummer. # MIT License # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal # in the Software without restriction, including without limitation the rights # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell # copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. import logging import re import gzip import statistics import platform from typing import Set # Create a logger logging.basicConfig(format="%(name)s - %(asctime)s %(levelname)s: %(message)s") logger = logging.getLogger("fasta_gtf_filter") logger.setLevel(logging.INFO) def format_yaml_like(data: dict, indent: int = 0) -> str: """Formats a dictionary to a YAML-like string. Args: data (dict): The dictionary to format. indent (int): The current indentation level. Returns: str: A string formatted as YAML. """ yaml_str = "" for key, value in data.items(): spaces = " " * indent if isinstance(value, dict): yaml_str += f"{spaces}{key}:\n{format_yaml_like(value, indent + 1)}" else: yaml_str += f"{spaces}{key}: {value}\n" return yaml_str def extract_fasta_seq_names(fasta_name: str) -> Set[str]: """Extracts the sequence names from a FASTA file.""" is_gz = fasta_name.endswith(".gz") open_fn = gzip.open if is_gz else open with open_fn(fasta_name) as fasta: sequences = set() for line in fasta: line = line.decode("utf-8") if is_gz else line if line.startswith(">"): sequences.add(line[1:].split(None, 1)[0]) return sequences def tab_delimited(file: str) -> float: """Check if file is tab-delimited and return median number of tabs.""" with open(file, "r") as f: data = f.read(102400) return statistics.median(line.count("\t") for line in data.split("\n")) def filter_gtf( fasta: str, gtf_in: str, filtered_gtf_out: str, skip_transcript_id_check: bool ) -> None: """Filter GTF file based on FASTA sequence names.""" if tab_delimited(gtf_in) != 8: raise ValueError("Invalid GTF file: Expected 9 tab-separated columns.") seq_names_in_genome = extract_fasta_seq_names(fasta) logger.info(f"Extracted chromosome sequence names from {fasta}") logger.debug( "All sequence IDs from FASTA: " + ", ".join(sorted(seq_names_in_genome)) ) seq_names_in_gtf = set() try: is_gz = gtf_in.endswith(".gz") open_fn = gzip.open if is_gz else open with open_fn(gtf_in) as gtf, open_fn(filtered_gtf_out, "wb" if is_gz else "w") as out: line_count = 0 for line in gtf: line = line.decode("utf-8") if is_gz else line seq_name = line.split("\t")[0] seq_names_in_gtf.add(seq_name) # Add sequence name to the set if seq_name in seq_names_in_genome: executor > local (25) [54/5d75e5] process > NFCORE_CIRCRNA:CIRCRNA:CAT_FASTQ (fust1_3) [100%] 6 of 6 ✔ [95/af2254] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:CLEAN_FASTA (fasta) [100%] 1 of 1 ✔ [86/e84479] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:GTFFILTER (gtf) [100%] 1 of 1, failed: 1 ✘ [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:UCSC_GTFTOGENEPRED - [79/5e62df] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:SEQKIT_SPLIT (fasta) [100%] 1 of 1 ✔ [17/cb8171] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:BOWTIE_BUILD (fasta.fa) [ 0%] 0 of 1 [1b/6b051c] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:BOWTIE2_BUILD (fasta.fa) [ 0%] 0 of 1 [bd/0bd0fd] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:BWA_INDEX (fasta.fa) [ 0%] 0 of 1 [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:HISAT2_EXTRACTSPLICESITES - [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:HISAT2_BUILD - [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:STAR_GENOMEGENERATE - [c9/d6fa7a] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:SAMTOOLS_FAIDX (fasta.fa) [100%] 1 of 1 ✔ [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:CIRCEXPLORER2_REFERENCE - [fe/a34870] process > NFCORE_CIRCRNA:CIRCRNA:FASTQC_TRIMGALORE:FASTQC (fust1_3) [100%] 6 of 6 ✔ [f8/721969] process > NFCORE_CIRCRNA:CIRCRNA:FASTQC_TRIMGALORE:TRIMGALORE (fust1_3) [100%] 6 of 6 ✔ [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:STAR2PASS:PASS_1 - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:STAR2PASS:SJDB - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:STAR2PASS:PASS_2 - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCEXPLORER2:PARSE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCEXPLORER2:ANNOTATE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCEXPLORER2:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCRNA_FINDER:MAIN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCRNA_FINDER:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:ALIGN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:SAMTOOLS_INDEX - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:SAMTOOLS_VIEW - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:ANCHORS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:MAIN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRIQUANT:MAIN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRIQUANT:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE1_1ST_PASS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE1_SJDB - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE1_2ND_PASS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE2_1ST_PASS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE2_SJDB - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE2_2ND_PASS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MAIN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:MAPSPLICE:ALIGN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:MAPSPLICE:PARSE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:MAPSPLICE:ANNOTATE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:MAPSPLICE:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:EXTRACT_COUNTS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:COMBINE_COUNTS_PER_TOOL - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FILTER_BSJS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:BED_ADD_SAMPLE_TOOL - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:COMBINE_TOOLS_PER_SAMPLE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:COMBINE_SAMPLES - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:INVESTIGATE_SHIFTS - Plus 45 more processes waiting for tasks… Staging foreign file: s3://ngi-igenomes/igenomes/Homo_sapiens/NCBI/GRCh38/Annotation/Genes/genes.gtf Execution cancelled -- Finishing pending tasks before exit ERROR ~ Error executing process > 'NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:GTFFILTER (gtf)' Caused by: Process `NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:GTFFILTER (gtf)` terminated with an error exit status (1) Command executed [/.nextflow/assets/nf-core/circrna/./workflows/circrna/../../subworkflows/local/../../modules/nf-core/custom/gtffilter/templates/gtffilter.py]: #!/usr/bin/env python # Written by Olga Botvinnik with subsequent reworking by Jonathan Manning and Nico Trummer. # MIT License # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal # in the Software without restriction, including without limitation the rights # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell # copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. import logging import re import gzip import statistics import platform from typing import Set # Create a logger logging.basicConfig(format="%(name)s - %(asctime)s %(levelname)s: %(message)s") logger = logging.getLogger("fasta_gtf_filter") logger.setLevel(logging.INFO) def format_yaml_like(data: dict, indent: int = 0) -> str: """Formats a dictionary to a YAML-like string. Args: data (dict): The dictionary to format. indent (int): The current indentation level. Returns: str: A string formatted as YAML. """ yaml_str = "" for key, value in data.items(): spaces = " " * indent if isinstance(value, dict): yaml_str += f"{spaces}{key}:\n{format_yaml_like(value, indent + 1)}" else: yaml_str += f"{spaces}{key}: {value}\n" return yaml_str def extract_fasta_seq_names(fasta_name: str) -> Set[str]: """Extracts the sequence names from a FASTA file.""" is_gz = fasta_name.endswith(".gz") open_fn = gzip.open if is_gz else open with open_fn(fasta_name) as fasta: sequences = set() for line in fasta: line = line.decode("utf-8") if is_gz else line if line.startswith(">"): sequences.add(line[1:].split(None, 1)[0]) return sequences def tab_delimited(file: str) -> float: """Check if file is tab-delimited and return median number of tabs.""" with open(file, "r") as f: data = f.read(102400) return statistics.median(line.count("\t") for line in data.split("\n")) def filter_gtf( fasta: str, gtf_in: str, filtered_gtf_out: str, skip_transcript_id_check: bool ) -> None: """Filter GTF file based on FASTA sequence names.""" if tab_delimited(gtf_in) != 8: raise ValueError("Invalid GTF file: Expected 9 tab-separated columns.") seq_names_in_genome = extract_fasta_seq_names(fasta) logger.info(f"Extracted chromosome sequence names from {fasta}") logger.debug( "All sequence IDs from FASTA: " + ", ".join(sorted(seq_names_in_genome)) ) seq_names_in_gtf = set() try: is_gz = gtf_in.endswith(".gz") open_fn = gzip.open if is_gz else open with open_fn(gtf_in) as gtf, open_fn(filtered_gtf_out, "wb" if is_gz else "w") as out: line_count = 0 for line in gtf: line = line.decode("utf-8") if is_gz else line seq_name = line.split("\t")[0] seq_names_in_gtf.add(seq_name) # Add sequence name to the set if seq_name in seq_names_in_genome: if skip_transcript_id_check or re.search( r'transcript_id "([^"]+)"', line ): executor > local (25) [54/5d75e5] process > NFCORE_CIRCRNA:CIRCRNA:CAT_FASTQ (fust1_3) [100%] 6 of 6 ✔ [95/af2254] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:CLEAN_FASTA (fasta) [100%] 1 of 1 ✔ [86/e84479] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:GTFFILTER (gtf) [100%] 1 of 1, failed: 1 ✘ [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:UCSC_GTFTOGENEPRED - [79/5e62df] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:SEQKIT_SPLIT (fasta) [100%] 1 of 1 ✔ [17/cb8171] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:BOWTIE_BUILD (fasta.fa) [ 0%] 0 of 1 [1b/6b051c] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:BOWTIE2_BUILD (fasta.fa) [ 0%] 0 of 1 [bd/0bd0fd] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:BWA_INDEX (fasta.fa) [ 0%] 0 of 1 [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:HISAT2_EXTRACTSPLICESITES - [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:HISAT2_BUILD - [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:STAR_GENOMEGENERATE - [c9/d6fa7a] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:SAMTOOLS_FAIDX (fasta.fa) [100%] 1 of 1 ✔ [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:CIRCEXPLORER2_REFERENCE - [fe/a34870] process > NFCORE_CIRCRNA:CIRCRNA:FASTQC_TRIMGALORE:FASTQC (fust1_3) [100%] 6 of 6 ✔ [f8/721969] process > NFCORE_CIRCRNA:CIRCRNA:FASTQC_TRIMGALORE:TRIMGALORE (fust1_3) [100%] 6 of 6 ✔ [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:STAR2PASS:PASS_1 - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:STAR2PASS:SJDB - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:STAR2PASS:PASS_2 - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCEXPLORER2:PARSE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCEXPLORER2:ANNOTATE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCEXPLORER2:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCRNA_FINDER:MAIN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCRNA_FINDER:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:ALIGN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:SAMTOOLS_INDEX - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:SAMTOOLS_VIEW - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:ANCHORS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:MAIN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRIQUANT:MAIN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRIQUANT:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE1_1ST_PASS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE1_SJDB - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE1_2ND_PASS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE2_1ST_PASS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE2_SJDB - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE2_2ND_PASS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MAIN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:MAPSPLICE:ALIGN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:MAPSPLICE:PARSE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:MAPSPLICE:ANNOTATE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:MAPSPLICE:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:EXTRACT_COUNTS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:COMBINE_COUNTS_PER_TOOL - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FILTER_BSJS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:BED_ADD_SAMPLE_TOOL - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:COMBINE_TOOLS_PER_SAMPLE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:COMBINE_SAMPLES - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:INVESTIGATE_SHIFTS - Plus 45 more processes waiting for tasks… Staging foreign file: s3://ngi-igenomes/igenomes/Homo_sapiens/NCBI/GRCh38/Annotation/Genes/genes.gtf Execution cancelled -- Finishing pending tasks before exit ERROR ~ Error executing process > 'NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:GTFFILTER (gtf)' Caused by: Process `NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:GTFFILTER (gtf)` terminated with an error exit status (1) Command executed [/.nextflow/assets/nf-core/circrna/./workflows/circrna/../../subworkflows/local/../../modules/nf-core/custom/gtffilter/templates/gtffilter.py]: #!/usr/bin/env python # Written by Olga Botvinnik with subsequent reworking by Jonathan Manning and Nico Trummer. # MIT License # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal # in the Software without restriction, including without limitation the rights # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell # copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. import logging import re import gzip import statistics import platform from typing import Set # Create a logger logging.basicConfig(format="%(name)s - %(asctime)s %(levelname)s: %(message)s") logger = logging.getLogger("fasta_gtf_filter") logger.setLevel(logging.INFO) def format_yaml_like(data: dict, indent: int = 0) -> str: """Formats a dictionary to a YAML-like string. Args: data (dict): The dictionary to format. indent (int): The current indentation level. Returns: str: A string formatted as YAML. """ yaml_str = "" for key, value in data.items(): spaces = " " * indent if isinstance(value, dict): yaml_str += f"{spaces}{key}:\n{format_yaml_like(value, indent + 1)}" else: yaml_str += f"{spaces}{key}: {value}\n" return yaml_str def extract_fasta_seq_names(fasta_name: str) -> Set[str]: """Extracts the sequence names from a FASTA file.""" is_gz = fasta_name.endswith(".gz") open_fn = gzip.open if is_gz else open with open_fn(fasta_name) as fasta: sequences = set() for line in fasta: line = line.decode("utf-8") if is_gz else line if line.startswith(">"): sequences.add(line[1:].split(None, 1)[0]) return sequences def tab_delimited(file: str) -> float: """Check if file is tab-delimited and return median number of tabs.""" with open(file, "r") as f: data = f.read(102400) return statistics.median(line.count("\t") for line in data.split("\n")) def filter_gtf( fasta: str, gtf_in: str, filtered_gtf_out: str, skip_transcript_id_check: bool ) -> None: """Filter GTF file based on FASTA sequence names.""" if tab_delimited(gtf_in) != 8: raise ValueError("Invalid GTF file: Expected 9 tab-separated columns.") seq_names_in_genome = extract_fasta_seq_names(fasta) logger.info(f"Extracted chromosome sequence names from {fasta}") logger.debug( "All sequence IDs from FASTA: " + ", ".join(sorted(seq_names_in_genome)) ) seq_names_in_gtf = set() try: is_gz = gtf_in.endswith(".gz") open_fn = gzip.open if is_gz else open with open_fn(gtf_in) as gtf, open_fn(filtered_gtf_out, "wb" if is_gz else "w") as out: line_count = 0 for line in gtf: line = line.decode("utf-8") if is_gz else line seq_name = line.split("\t")[0] seq_names_in_gtf.add(seq_name) # Add sequence name to the set if seq_name in seq_names_in_genome: if skip_transcript_id_check or re.search( r'transcript_id "([^"]+)"', line ): out.write(line.encode() if is_gz else line) executor > local (25) [54/5d75e5] process > NFCORE_CIRCRNA:CIRCRNA:CAT_FASTQ (fust1_3) [100%] 6 of 6 ✔ [95/af2254] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:CLEAN_FASTA (fasta) [100%] 1 of 1 ✔ [86/e84479] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:GTFFILTER (gtf) [100%] 1 of 1, failed: 1 ✘ [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:UCSC_GTFTOGENEPRED - [79/5e62df] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:SEQKIT_SPLIT (fasta) [100%] 1 of 1 ✔ [17/cb8171] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:BOWTIE_BUILD (fasta.fa) [ 0%] 0 of 1 [1b/6b051c] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:BOWTIE2_BUILD (fasta.fa) [ 0%] 0 of 1 [bd/0bd0fd] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:BWA_INDEX (fasta.fa) [ 0%] 0 of 1 [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:HISAT2_EXTRACTSPLICESITES - [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:HISAT2_BUILD - [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:STAR_GENOMEGENERATE - [c9/d6fa7a] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:SAMTOOLS_FAIDX (fasta.fa) [100%] 1 of 1 ✔ [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:CIRCEXPLORER2_REFERENCE - [fe/a34870] process > NFCORE_CIRCRNA:CIRCRNA:FASTQC_TRIMGALORE:FASTQC (fust1_3) [100%] 6 of 6 ✔ [f8/721969] process > NFCORE_CIRCRNA:CIRCRNA:FASTQC_TRIMGALORE:TRIMGALORE (fust1_3) [100%] 6 of 6 ✔ [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:STAR2PASS:PASS_1 - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:STAR2PASS:SJDB - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:STAR2PASS:PASS_2 - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCEXPLORER2:PARSE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCEXPLORER2:ANNOTATE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCEXPLORER2:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCRNA_FINDER:MAIN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCRNA_FINDER:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:ALIGN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:SAMTOOLS_INDEX - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:SAMTOOLS_VIEW - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:ANCHORS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:MAIN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRIQUANT:MAIN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRIQUANT:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE1_1ST_PASS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE1_SJDB - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE1_2ND_PASS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE2_1ST_PASS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE2_SJDB - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE2_2ND_PASS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MAIN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:MAPSPLICE:ALIGN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:MAPSPLICE:PARSE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:MAPSPLICE:ANNOTATE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:MAPSPLICE:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:EXTRACT_COUNTS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:COMBINE_COUNTS_PER_TOOL - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FILTER_BSJS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:BED_ADD_SAMPLE_TOOL - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:COMBINE_TOOLS_PER_SAMPLE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:COMBINE_SAMPLES - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:INVESTIGATE_SHIFTS - Plus 45 more processes waiting for tasks… Staging foreign file: s3://ngi-igenomes/igenomes/Homo_sapiens/NCBI/GRCh38/Annotation/Genes/genes.gtf Execution cancelled -- Finishing pending tasks before exit ERROR ~ Error executing process > 'NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:GTFFILTER (gtf)' Caused by: Process `NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:GTFFILTER (gtf)` terminated with an error exit status (1) Command executed [/.nextflow/assets/nf-core/circrna/./workflows/circrna/../../subworkflows/local/../../modules/nf-core/custom/gtffilter/templates/gtffilter.py]: #!/usr/bin/env python # Written by Olga Botvinnik with subsequent reworking by Jonathan Manning and Nico Trummer. # MIT License # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal # in the Software without restriction, including without limitation the rights # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell # copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. import logging import re import gzip import statistics import platform from typing import Set # Create a logger logging.basicConfig(format="%(name)s - %(asctime)s %(levelname)s: %(message)s") logger = logging.getLogger("fasta_gtf_filter") logger.setLevel(logging.INFO) def format_yaml_like(data: dict, indent: int = 0) -> str: """Formats a dictionary to a YAML-like string. Args: data (dict): The dictionary to format. indent (int): The current indentation level. Returns: str: A string formatted as YAML. """ yaml_str = "" for key, value in data.items(): spaces = " " * indent if isinstance(value, dict): yaml_str += f"{spaces}{key}:\n{format_yaml_like(value, indent + 1)}" else: yaml_str += f"{spaces}{key}: {value}\n" return yaml_str def extract_fasta_seq_names(fasta_name: str) -> Set[str]: """Extracts the sequence names from a FASTA file.""" is_gz = fasta_name.endswith(".gz") open_fn = gzip.open if is_gz else open with open_fn(fasta_name) as fasta: sequences = set() for line in fasta: line = line.decode("utf-8") if is_gz else line if line.startswith(">"): sequences.add(line[1:].split(None, 1)[0]) return sequences def tab_delimited(file: str) -> float: """Check if file is tab-delimited and return median number of tabs.""" with open(file, "r") as f: data = f.read(102400) return statistics.median(line.count("\t") for line in data.split("\n")) def filter_gtf( fasta: str, gtf_in: str, filtered_gtf_out: str, skip_transcript_id_check: bool ) -> None: """Filter GTF file based on FASTA sequence names.""" if tab_delimited(gtf_in) != 8: raise ValueError("Invalid GTF file: Expected 9 tab-separated columns.") seq_names_in_genome = extract_fasta_seq_names(fasta) logger.info(f"Extracted chromosome sequence names from {fasta}") logger.debug( "All sequence IDs from FASTA: " + ", ".join(sorted(seq_names_in_genome)) ) seq_names_in_gtf = set() try: is_gz = gtf_in.endswith(".gz") open_fn = gzip.open if is_gz else open with open_fn(gtf_in) as gtf, open_fn(filtered_gtf_out, "wb" if is_gz else "w") as out: line_count = 0 for line in gtf: line = line.decode("utf-8") if is_gz else line seq_name = line.split("\t")[0] seq_names_in_gtf.add(seq_name) # Add sequence name to the set if seq_name in seq_names_in_genome: if skip_transcript_id_check or re.search( r'transcript_id "([^"]+)"', line ): out.write(line.encode() if is_gz else line) line_count += 1 if line_count == 0: raise ValueError("All GTF lines removed by filters") except IOError as e: logger.error(f"File operation failed: {e}") executor > local (25) [54/5d75e5] process > NFCORE_CIRCRNA:CIRCRNA:CAT_FASTQ (fust1_3) [100%] 6 of 6 ✔ [95/af2254] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:CLEAN_FASTA (fasta) [100%] 1 of 1 ✔ [86/e84479] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:GTFFILTER (gtf) [100%] 1 of 1, failed: 1 ✘ [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:UCSC_GTFTOGENEPRED - [79/5e62df] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:SEQKIT_SPLIT (fasta) [100%] 1 of 1 ✔ [17/cb8171] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:BOWTIE_BUILD (fasta.fa) [ 0%] 0 of 1 [1b/6b051c] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:BOWTIE2_BUILD (fasta.fa) [ 0%] 0 of 1 [bd/0bd0fd] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:BWA_INDEX (fasta.fa) [ 0%] 0 of 1 [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:HISAT2_EXTRACTSPLICESITES - [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:HISAT2_BUILD - [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:STAR_GENOMEGENERATE - [c9/d6fa7a] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:SAMTOOLS_FAIDX (fasta.fa) [100%] 1 of 1 ✔ [- ] process > NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:CIRCEXPLORER2_REFERENCE - [fe/a34870] process > NFCORE_CIRCRNA:CIRCRNA:FASTQC_TRIMGALORE:FASTQC (fust1_3) [100%] 6 of 6 ✔ [f8/721969] process > NFCORE_CIRCRNA:CIRCRNA:FASTQC_TRIMGALORE:TRIMGALORE (fust1_3) [100%] 6 of 6 ✔ [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:STAR2PASS:PASS_1 - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:STAR2PASS:SJDB - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:STAR2PASS:PASS_2 - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCEXPLORER2:PARSE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCEXPLORER2:ANNOTATE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCEXPLORER2:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCRNA_FINDER:MAIN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRCRNA_FINDER:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:ALIGN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:SAMTOOLS_INDEX - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:SAMTOOLS_VIEW - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:ANCHORS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:MAIN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FIND_CIRC:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRIQUANT:MAIN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:CIRIQUANT:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE1_1ST_PASS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE1_SJDB - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE1_2ND_PASS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE2_1ST_PASS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE2_SJDB - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MATE2_2ND_PASS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:MAIN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:DCC:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:MAPSPLICE:ALIGN - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:MAPSPLICE:PARSE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:MAPSPLICE:ANNOTATE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:MAPSPLICE:UNIFY - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:EXTRACT_COUNTS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:COMBINE_COUNTS_PER_TOOL - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:FILTER_BSJS - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:BED_ADD_SAMPLE_TOOL - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:COMBINE_TOOLS_PER_SAMPLE - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:COMBINE_SAMPLES - [- ] process > NFCORE_CIRCRNA:CIRCRNA:BSJ_DETECTION:INVESTIGATE_SHIFTS - Plus 45 more processes waiting for tasks… Staging foreign file: s3://ngi-igenomes/igenomes/Homo_sapiens/NCBI/GRCh38/Annotation/Genes/genes.gtf Execution cancelled -- Finishing pending tasks before exit ERROR ~ Error executing process > 'NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:GTFFILTER (gtf)' Caused by: Process `NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:GTFFILTER (gtf)` terminated with an error exit status (1) Command executed [/.nextflow/assets/nf-core/circrna/./workflows/circrna/../../subworkflows/local/../../modules/nf-core/custom/gtffilter/templates/gtffilter.py]: #!/usr/bin/env python # Written by Olga Botvinnik with subsequent reworking by Jonathan Manning and Nico Trummer. # MIT License # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal # in the Software without restriction, including without limitation the rights # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell # copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. import logging import re import gzip import statistics import platform from typing import Set # Create a logger logging.basicConfig(format="%(name)s - %(asctime)s %(levelname)s: %(message)s") logger = logging.getLogger("fasta_gtf_filter") logger.setLevel(logging.INFO) def format_yaml_like(data: dict, indent: int = 0) -> str: """Formats a dictionary to a YAML-like string. Args: data (dict): The dictionary to format. indent (int): The current indentation level. Returns: str: A string formatted as YAML. """ yaml_str = "" for key, value in data.items(): spaces = " " * indent if isinstance(value, dict): yaml_str += f"{spaces}{key}:\n{format_yaml_like(value, indent + 1)}" else: yaml_str += f"{spaces}{key}: {value}\n" return yaml_str def extract_fasta_seq_names(fasta_name: str) -> Set[str]: """Extracts the sequence names from a FASTA file.""" is_gz = fasta_name.endswith(".gz") open_fn = gzip.open if is_gz else open with open_fn(fasta_name) as fasta: sequences = set() for line in fasta: line = line.decode("utf-8") if is_gz else line if line.startswith(">"): sequences.add(line[1:].split(None, 1)[0]) return sequences def tab_delimited(file: str) -> float: """Check if file is tab-delimited and return median number of tabs.""" with open(file, "r") as f: data = f.read(102400) return statistics.median(line.count("\t") for line in data.split("\n")) def filter_gtf( fasta: str, gtf_in: str, filtered_gtf_out: str, skip_transcript_id_check: bool ) -> None: """Filter GTF file based on FASTA sequence names.""" if tab_delimited(gtf_in) != 8: raise ValueError("Invalid GTF file: Expected 9 tab-separated columns.") seq_names_in_genome = extract_fasta_seq_names(fasta) logger.info(f"Extracted chromosome sequence names from {fasta}") logger.debug( "All sequence IDs from FASTA: " + ", ".join(sorted(seq_names_in_genome)) ) seq_names_in_gtf = set() try: is_gz = gtf_in.endswith(".gz") open_fn = gzip.open if is_gz else open with open_fn(gtf_in) as gtf, open_fn(filtered_gtf_out, "wb" if is_gz else "w") as out: line_count = 0 for line in gtf: line = line.decode("utf-8") if is_gz else line seq_name = line.split("\t")[0] seq_names_in_gtf.add(seq_name) # Add sequence name to the set if seq_name in seq_names_in_genome: if skip_transcript_id_check or re.search( r'transcript_id "([^"]+)"', line ): out.write(line.encode() if is_gz else line) line_count += 1 if line_count == 0: raise ValueError("All GTF lines removed by filters") except IOError as e: logger.error(f"File operation failed: {e}") return logger.debug("All sequence IDs from GTF: " + ", ".join(sorted(seq_names_in_gtf))) logger.info( f"Extracted {line_count} matching sequences from {gtf_in} into {filtered_gtf_out}" ) filter_gtf("fasta.fa", "genes.gtf", "gtf.filtered.gtf", False) # Versions versions = {"NFCORE_CIRCRNA:CIRCRNA:PREPARE_GENOME:GTFFILTER": {"python": platform.python_version()}} with open("versions.yml", "w") as f: f.write(format_yaml_like(versions)) Command exit status: 1 Command output: (empty) Command error: fasta_gtf_filter - 2025-01-20 13:36:06,475 INFO: Extracted chromosome sequence names from fasta.fa Traceback (most recent call last): File ".command.sh", line 125, in filter_gtf("fasta.fa", "genes.gtf", "gtf.filtered.gtf", False) File ".command.sh", line 113, in filter_gtf raise ValueError("All GTF lines removed by filters") ValueError: All GTF lines removed by filters Work dir: /home/cloetel/workspace/circrna/work/86/e844790f4b87454df368ca6afbf11d Container: quay.io/biocontainers/python:3.9--1 Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line -- Check '.nextflow.log' file for details ERROR ~ Pipeline failed. Please refer to troubleshooting docs: https://nf-co.re/docs/usage/troubleshooting -- Check '.nextflow.log' file for details No transcriptome fasta file produced. No circular RNAs were found by at least 3 tools and in at least 1 samples. These thresholds can be adjusted using the parameters 'min_tools' and 'min_samples'. Feel free to check the preliminary results in 'results' You can enable saving intermediate files by setting the parameter 'save_intermediates' to 'true'. ERROR ~ Pipeline failed. Please refer to troubleshooting docs: https://nf-co.re/docs/usage/troubleshooting -- Check '.nextflow.log' file for details No circular RNAs were found by at least 3 tools and in at least 1 samples. These thresholds can be adjusted using the parameters 'min_tools' and 'min_samples'. Feel free to check the preliminary results in 'results' You can enable saving intermediate files by setting the parameter 'save_intermediates' to 'true'. ERROR ~ A DataflowVariable can only be assigned once. Use bind() to allow for equal values to be passed into already-bound variables. -- Check '.nextflow.log' file for details WARN: [nf-core/circrna] No reports found from process 'MULTIQC' ERROR ~ Failed to invoke `workflow.onComplete` event handler -- Check script '/.nextflow/assets/nf-core/circrna/./subworkflows/local/utils_nfcore_circrna_pipeline/main.nf' at line: 133 or see '.nextflow.log' file for more details WARN: Killing running tasks (3)