Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

This class needs to be public so surefire don't crash running the tests #353

Open
wants to merge 44 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
9e24125
This class needs to be public so surefire don't crash running the tests
GuillaumeFromage Dec 20, 2024
f0d0fe8
Dumb fixes so the test passes for compilation. This might not be a go…
GuillaumeFromage Dec 22, 2024
9cd1d35
Merge branch 'master' of ssh://github.com/GuillaumeFromage/catma
GuillaumeFromage Dec 22, 2024
07569d9
Removed mandatory recaptcha
GuillaumeFromage Dec 27, 2024
a10b823
Relaxed requirements on email confirmation
GuillaumeFromage Dec 27, 2024
e6041cf
Added more extension install instructions
GuillaumeFromage Dec 27, 2024
2202d33
Fucking around, WIP, trying to get some combobox for parent tag
GuillaumeFromage Dec 30, 2024
f3c2d16
This reverts a change in 2202d3343365bec37c59ddd5abc1dc47974c62d9 ;
GuillaumeFromage Dec 30, 2024
b6fc937
When a user add a tag while selecting a tag instead of a tagset, it a…
GuillaumeFromage Dec 30, 2024
77f827f
Added some documentation
GuillaumeFromage Dec 30, 2024
00c2eb9
Set the right value for show parent and show tagset
GuillaumeFromage Dec 30, 2024
bfd8172
Break down the initComponent function to allow for submission of a li…
GuillaumeFromage Dec 30, 2024
1865a71
Allow selection of parent tag in the user interface
GuillaumeFromage Dec 30, 2024
5d74c52
Made a function to recursively draw the subtags hierarchy
GuillaumeFromage Dec 31, 2024
d0f8146
Make the add subtag dialog only show the tagsets shown in the parent …
GuillaumeFromage Dec 31, 2024
9df78cb
Allow interactive selection of parents when adding a subtag
GuillaumeFromage Dec 31, 2024
3e2a5e0
The UI for editing tags with the parent tag and the tagset is almost …
GuillaumeFromage Dec 31, 2024
4e51006
Properly display parent and tagset selection boxes in edit tag dialog
GuillaumeFromage Jan 1, 2025
8fe75b6
Made the change to the parent tag sent down to the backend
GuillaumeFromage Jan 1, 2025
2375388
Attempt to fix redrawing issue when changing a TagDefinition in the h…
GuillaumeFromage Jan 9, 2025
c4a9ef9
Set the new value for the parent uuid for when it was modified in the UI
GuillaumeFromage Jan 9, 2025
44f31cd
Fire a specific event for when a tag is moved in the hierarchy
GuillaumeFromage Jan 9, 2025
8f5f645
Added a listener for when an object moved in the hierarchy and helper…
GuillaumeFromage Jan 9, 2025
06d6899
Added a function that handles the changes in hierarchy of a TagDefini…
GuillaumeFromage Jan 9, 2025
fe52ac6
First part of the moving object around code
GuillaumeFromage Jan 9, 2025
2a2d172
Added code for the corner cases of moving the hierarchy around, added…
GuillaumeFromage Jan 9, 2025
235ea05
Fixed an horrible typo and first stab at getting the UI properly foll…
GuillaumeFromage Jan 9, 2025
efa8897
Added a note of an important task (making sure the users can't unroot…
GuillaumeFromage Jan 9, 2025
ce9735b
Added a note of an important task (making sure the users can't unroot…
GuillaumeFromage Jan 9, 2025
d3773f3
Fix a typo that crash the UI
GuillaumeFromage Jan 10, 2025
7617d5f
Merge branch 'master' of ssh://github.com/GuillaumeFromage/catma
GuillaumeFromage Jan 10, 2025
88e2eeb
Moving item in the hierarchy works in a tagset.
GuillaumeFromage Jan 10, 2025
e546c0a
I've made an independant listener
GuillaumeFromage Jan 10, 2025
5061c3b
Disable the possibility of saving a tag in a different tagset
GuillaumeFromage Jan 10, 2025
929deb8
Fix the UI
GuillaumeFromage Jan 14, 2025
f544142
Don't show the item and its subitem in the parent selection field of …
GuillaumeFromage Jan 14, 2025
29a27ec
Removed debug code
GuillaumeFromage Jan 16, 2025
eea758e
We don't need to set the tag deleted, it can be fetched right away by…
GuillaumeFromage Jan 16, 2025
4078135
We use a custom function to do the change so we can preserve the chil…
GuillaumeFromage Jan 16, 2025
b7fe478
Created a function that move a tagDefinition in the hierarchy
GuillaumeFromage Jan 16, 2025
5da5a8a
Copied from TagsView the proper way to move items
GuillaumeFromage Jan 16, 2025
b4c4a38
Added a listener so when the data change, it gets passed down to the …
GuillaumeFromage Jan 16, 2025
1c16cc7
Made the hierarchy change function
GuillaumeFromage Jan 16, 2025
3122602
Select a parent tagset even when none is selected
GuillaumeFromage Jan 17, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 87 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,87 @@
# Introduction

CATMA is a web based application for text annotation and analysis.

It has two main components: an annotation component that allows the user to annotate text and an analysis component that supports pattern analysis of text and annotations in combination through a custom but easy query language and a data visualization facility based on [Vega](https://vega.github.io/vega/).

It also allows the project-centric management of text corpora, annotation collections, tagsets and team members.

CATMA uses a Gitlab instance as its backend to store and manage the project's resources and team. All resources are versioned and available through either the CATMA web interface or the Gitlab REST API.

# Installation & Setup

CATMA uses a stock Gitlab installation with the following mandatory configuration:

The default dev ops pipeline needs to be switched off.
The default branch protection needs to be "partially protected".

The CATMA war file can be build with
`mvn clean compile package`

CATMA is tested with the jetty servlet container.

Copy and configure the catma.properties file from the resource folder to the web app root folder.
# Introduction

CATMA is a web based application for text annotation and analysis.

It has two main components: an annotation component that allows the user to annotate text and an analysis component that supports pattern analysis of text and annotations in combination through a custom but easy query language and a data visualization facility based on [Vega](https://vega.github.io/vega/).

It also allows the project-centric management of text corpora, annotation collections, tagsets and team members.

CATMA uses a Gitlab instance as its backend to store and manage the project's resources and team. All resources are versioned and available through either the CATMA web interface or the Gitlab REST API.

# Requirements
* jdk11 or lower, ideally jdk8
* say on debian 12/bookworm, you wanna get jdk from oldstable, so you don't have to change libc6
* maven
* docker

# Installation & Setup
First, you need to get some dependencies dealt with:
```
git clone https://github.com/forTEXT/serverside-elements.git # some bundled dependency that is patched
cd serverside-elements
mvn compile package
mvn install:install-file -Dfile=elements/target/elements-0.2.3-CATMA.jar -DpomFile=elements/pom.xml
cd ..
git clone https://github.com/forTEXT/gitlab4j-api.git # some other patched dependency
cd gitlab4j-api
mvn compile package
mvn install:install-file -Dfile=target/gitlab4j-api-5.1.0-SNAPSHOT.jar -DpomFile=pom.xml
cd ..
sudo docker pull gitlab/gitlab-ce
echo > docker-compose.yml """
services:
gitlab-server:
image: 'gitlab/gitlab-ce'
container_name: gitlab-server
ports:
- '8088:80'
environment:
GITLAB_ROOT_EMAIL: '[email protected]'
GITLAB_ROOT_PASSWORD: 'Abcd@0123456789 CHANGE THIS'
volumes:
- ./gitlab/config:/etc/gitlab
- ./gitlab/data:/var/opt/gitlab
"""
sudo docker compose up
```

This has downloaded a gitlab docker insancen. When the virtual machine in which gitlab
runs will be started you should be able to configure in your browser at localhost:8088.
You then need to change the configuration this way:
* The default dev ops pipeline needs to be switched off.
* That is in settings => CI/CD => Continuous integration and development => remove
the tickmark before "Default to Auto DevOps pipeline for all projects"
* The default branch protection needs to be disabled
* Settings => Repository => Default branch => Initial branch protection needs to be
set to not protected
* Then you need to go in your user settings and create and access token for the API
for the root account
* User settings => access tokens

```
git clone https://github.com/forTEXT/catma
cd catma
cp src/main/resources/catma.properties .
```

You probably wanna change some config option in catma.properties ; namely, if you
plan on just running catma via maven for testing purposes, you probably want
to set the BASE_URL to http://localhost:8080/catma/. Also you need to configure the
gitlab access token and server url.

Then you can:
```
mvn clean compile package
```
As maven runs tests after the compilation, the gitlab link gets extensively tested,
so you need that link to run so you can compile the catma jar file.

Copy and configure the catma.properties file from the resource folder to the web app root folder.
```
cp catma.properties src/main/webapp/catma.properties
```

CATMA is tested with the jetty servlet container. To run it, you just have to type:
```
mvn jetty:run
```
18 changes: 18 additions & 0 deletions src/main/java/de/catma/repository/git/GitProjectHandler.java
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,24 @@ public String createOrUpdateTag(String tagsetId, TagDefinition tagDefinition, St
}
}

public String moveTagAndUpdateAnnotations(TagsetDefinition tsdFrom, TagsetDefinition tsdTo, TagDefinition tdFrom, TagDefinition tdTo, Multimap<String, TagInstance> tagInstancesByCollectionId, String commitMsg) throws IOException {
try (LocalGitRepositoryManager localGitRepoManager = localGitRepositoryManager) {
localGitRepoManager.open(projectReference.getNamespace(), projectReference.getProjectId());
GitTagsetHandler gitTagsetHandler = new GitTagsetHandler(
localGitRepoManager,
projectPath,
remoteGitServerManager.getUsername(),
remoteGitServerManager.getEmail()
);
String projectRevision = gitTagsetHandler.moveTagDefinition(tdFrom, tdTo, commitMsg);

localGitRepoManager.push(jGitCredentialsManager);

return projectRevision;
}
}


public String removeTagAndAnnotations(TagDefinition tagDefinition, Multimap<String, TagInstance> tagInstancesByCollectionId) throws IOException {
try (LocalGitRepositoryManager localGitRepoManager = localGitRepositoryManager) {
localGitRepoManager.open(projectReference.getNamespace(), projectReference.getProjectId());
Expand Down
74 changes: 74 additions & 0 deletions src/main/java/de/catma/repository/git/GitTagsetHandler.java
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,77 @@ public ContentInfoSet getContentInfoSet(String tagsetId) throws IOException {

return contentInfoSet;
}

/**
* Moves a tag definition between one or multiple tagsets identified by <code>tagsetId</code>.
*
* @param tdFrom {@link TagDefinition} source
* @param tdTo {@link TagDefinition} destination
* @param commitMsg commit message
* @return the new project revision hash
* @throws IOException if an error occurs during the move of the tag definition
*/
public String moveTagDefinition(
TagDefinition tdFrom,
TagDefinition tdTo,
String commitMsg
) throws IOException {
String pdj = "/propertydefs.json";
String common = String.format("%s/%s/", this.projectDirectory.getAbsolutePath(), GitProjectHandler.TAGSETS_DIRECTORY_NAME);
String fromTagsetDir = String.format("%s/%s/", common, tdFrom.getTagsetDefinitionUuid());
String toTagsetDir = String.format("%s/%s/", common, tdTo.getTagsetDefinitionUuid());

/* This work like it seems: all the folders having subdirectories have a folder
at the root of the tagset folder containing the subitems. This mean that the
hiearchy isn't stored as it is in the UI. */
String fromPath = fromTagsetDir +
(StringUtils.isEmpty(tdFrom.getParentUuid()) ? "" : (tdFrom.getParentUuid() + "/"))
+ tdFrom.getUuid();
String toPath = toTagsetDir +
(StringUtils.isEmpty(tdTo.getParentUuid()) ? "" : (tdTo.getParentUuid() + "/"))
+ tdTo.getUuid();
File from = Paths.get(fromPath).toFile();
File to = Paths.get(toPath).toFile();
if ((StringUtils.isEmpty(tdFrom.getParentUuid()) && StringUtils.isEmpty(tdTo.getParentUuid())) ||
(!StringUtils.isEmpty(tdFrom.getParentUuid()) && !StringUtils.isEmpty(tdTo.getParentUuid()))) {
/* if we move from an item in a tree somewhere to an item in a tree somewhere else, or if we
move a parent around in the root of the hierarchy, its fine, we just move the folder
around */
FileUtils.moveDirectory(from, to);
this.localGitRepositoryManager.remove(Paths.get(fromPath + pdj).toFile());
this.localGitRepositoryManager.remove(from);
}
if (StringUtils.isEmpty(tdFrom.getParentUuid()) && !StringUtils.isEmpty(tdTo.getParentUuid())) {
/* if we moved from the root to an object lower in the hierarchy, we need to bring the children back down
(if it has any) */
/* There shouldn't be a subdir with the name of the current tag UUID in its parent folder */
to.mkdirs();
FileUtils.moveFile(Paths.get(fromPath+pdj).toFile(), Paths.get(toPath+pdj).toFile());
/* That leaves the children in the root of the tagset under the directory with the name of
their parent, which is what we wanted anyways */
this.localGitRepositoryManager.remove(Paths.get(fromPath + pdj).toFile());
}
if (StringUtils.isEmpty(tdTo.getParentUuid()) && !StringUtils.isEmpty(tdFrom.getParentUuid())) {
/* If we moved back to the root, our children are already in a directory called with our UUID at the root,
we need to not squish them, so we only move the propertydefs there */
FileUtils.moveFile(Paths.get(fromPath+pdj).toFile(), Paths.get(toPath+pdj).toFile());
this.localGitRepositoryManager.remove(Paths.get(fromPath + pdj).toFile());
}

GitTagDefinition gitTagDefinition = new GitTagDefinition(tdTo);
String serializedGitTagDefinition =
new SerializationHelper<GitTagDefinition>().serialize(gitTagDefinition);

String projectRevision = this.localGitRepositoryManager.addAndCommit(
Paths.get(toPath + pdj).toFile(),
serializedGitTagDefinition.getBytes(StandardCharsets.UTF_8),
commitMsg,
this.username,
this.email);
System.out.println("Deletion test commit:" + projectRevision);
return projectRevision;
}


/**
* Creates a tag definition within the tagset identified by <code>tagsetId</code>.
Expand All @@ -187,6 +258,9 @@ public String createOrUpdateTagDefinition(
String commitMsg
) throws IOException {

/* This work like it seems: all the folders having subdirectories have a folder
at the root of the tagset folder containing the subitems. This mean that the
hiearchy isn't stored as it is in the UI. */
String targetPropertyDefinitionsFileRelativePath =
(StringUtils.isEmpty(tagDefinition.getParentUuid()) ? "" : (tagDefinition.getParentUuid() + "/"))
+ tagDefinition.getUuid()
Expand Down
81 changes: 71 additions & 10 deletions src/main/java/de/catma/repository/git/GraphWorktreeProject.java
Original file line number Diff line number Diff line change
Expand Up @@ -409,6 +409,30 @@ else if (evt.getNewValue() == null) { // TagDefinition was deleted
};
tagManager.addPropertyChangeListener(TagManagerEvent.tagDefinitionChanged, tagDefinitionChangedListener);

PropertyChangeListener tagDefinitionMovedListener = new PropertyChangeListener() {
public void propertyChange(final PropertyChangeEvent evt) {
if (!tagManagerListenersEnabled) {
return;
}

try {
@SuppressWarnings("unchecked")
final Pair<TagsetDefinition, TagDefinition> newVal = (Pair<TagsetDefinition, TagDefinition>) evt.getNewValue();
final Pair<TagsetDefinition, TagDefinition> oldVal = (Pair<TagsetDefinition, TagDefinition>) evt.getOldValue();
moveTagDefinition((TagsetDefinition)(oldVal.getFirst()), (TagsetDefinition)(newVal.getFirst()), (TagDefinition)(oldVal.getSecond()), (TagDefinition)(newVal.getSecond()));
}
catch (Exception e) {
propertyChangeSupport.firePropertyChange(
ProjectEvent.exceptionOccurred.name(),
null,
e
);
}
}
};

tagManager.addPropertyChangeListener(TagManagerEvent.tagDefinitionMoved, tagDefinitionMovedListener);

PropertyChangeListener userDefinedPropertyChangedListener = new PropertyChangeListener() {
public void propertyChange(PropertyChangeEvent evt) {
if (!tagManagerListenersEnabled) {
Expand Down Expand Up @@ -652,26 +676,27 @@ private void removeTagsetDefinition(TagsetDefinition tagsetDefinition) throws Ex
graphProjectHandler.updateProjectRevision(oldRootRevisionHash, rootRevisionHash);
}

private void addTagDefinition(TagDefinition tagDefinition, TagsetDefinition tagsetDefinition) throws Exception {
private void updateTagDefinition(TagDefinition tagDefinition, TagsetDefinition tagsetDefinition) throws Exception {
if (isReadOnly()) {
throw new IllegalStateException(
String.format(
"Project \"%1$s\" is in read-only mode! Cannot add tag \"%2$s\" to tagset \"%3$s\".",
"Project \"%1$s\" is in read-only mode! Cannot update tag \"%2$s\" with ID %3$s in tagset \"%4$s\".",
projectReference.getName(),
tagDefinition.getName(),
tagDefinition.getUuid(),
tagsetDefinition.getName()
)
);
}

String oldRootRevisionHash = rootRevisionHash;

// create tag in repo and commit
// update tag in repo and commit
rootRevisionHash = gitProjectHandler.createOrUpdateTag(
tagsetDefinition.getUuid(),
tagDefinition,
String.format(
"Added tag \"%1$s\" with ID %2$s to tagset \"%3$s\" with ID %4$s",
"Updated tag \"%1$s\" with ID %2$s in tagset \"%3$s\" with ID %4$s",
tagDefinition.getName(),
tagDefinition.getUuid(),
tagsetDefinition.getName(),
Expand All @@ -683,27 +708,25 @@ private void addTagDefinition(TagDefinition tagDefinition, TagsetDefinition tags
graphProjectHandler.updateProjectRevision(oldRootRevisionHash, rootRevisionHash);
}

private void updateTagDefinition(TagDefinition tagDefinition, TagsetDefinition tagsetDefinition) throws Exception {
private void addTagDefinition(TagDefinition tagDefinition, TagsetDefinition tagsetDefinition) throws Exception {
if (isReadOnly()) {
throw new IllegalStateException(
String.format(
"Project \"%1$s\" is in read-only mode! Cannot update tag \"%2$s\" with ID %3$s in tagset \"%4$s\".",
"Project \"%1$s\" is in read-only mode! Cannot add tag \"%2$s\" to tagset \"%3$s\".",
projectReference.getName(),
tagDefinition.getName(),
tagDefinition.getUuid(),
tagsetDefinition.getName()
)
);
}

String oldRootRevisionHash = rootRevisionHash;

// update tag in repo and commit
// create tag in repo and commit
rootRevisionHash = gitProjectHandler.createOrUpdateTag(
tagsetDefinition.getUuid(),
tagDefinition,
String.format(
"Updated tag \"%1$s\" with ID %2$s in tagset \"%3$s\" with ID %4$s",
"Added tag \"%1$s\" with ID %2$s to tagset \"%3$s\" with ID %4$s",
tagDefinition.getName(),
tagDefinition.getUuid(),
tagsetDefinition.getName(),
Expand All @@ -715,6 +738,44 @@ private void updateTagDefinition(TagDefinition tagDefinition, TagsetDefinition t
graphProjectHandler.updateProjectRevision(oldRootRevisionHash, rootRevisionHash);
}

private void moveTagDefinition(TagsetDefinition tsdFrom, TagsetDefinition tsdTo, TagDefinition tdFrom, TagDefinition tdTo) throws Exception {
if (isReadOnly()) {
throw new IllegalStateException(
String.format(
"Project \"%1$s\" is in read-only mode! Cannot move tag \"%2$s\" with ID %3$s from tagset \"%4$s\".",
projectReference.getName(),
tdTo.getName(),
tdTo.getUuid(),
tsdTo.getName()
)
);
}

String oldRootRevisionHash = rootRevisionHash;

Multimap<String, TagReference> tagReferencesByCollectionId = graphProjectHandler.getTagReferencesByCollectionId(tdFrom);
Multimap<String, TagInstance> tagInstancesByCollectionId = Multimaps.transformValues(
tagReferencesByCollectionId, TagReference::getTagInstance
);

// update tag in repo and commit
rootRevisionHash = gitProjectHandler.moveTagAndUpdateAnnotations(
tsdFrom, tsdTo, tdFrom, tdTo,
tagInstancesByCollectionId,
String.format(
"Moved tag \"%1$s\" with ID %2$s from tagset \"%3$s\" with ID %4$s to tagset \"%5$s\" with id %6$s",
tdTo.getName(),
tdTo.getUuid(),
tsdFrom.getName(),
tsdFrom.getUuid(),
tsdTo.getName(),
tsdTo.getUuid()
)
);
// update revision hash on GraphProjectHandler
graphProjectHandler.updateProjectRevision(oldRootRevisionHash, rootRevisionHash);
}

private void removeTagDefinition(TagDefinition tagDefinition, TagsetDefinition tagsetDefinition) throws Exception {
if (isReadOnly()) {
throw new IllegalStateException(
Expand Down
Loading