Suppose you are building a platform, that integrates data of sensitive nature. Such a platform can be an IT-solution, like a web application, for clinical studies that enables patients and doctors to login and access the data, such as findings, demographic data or enter new information. Another example would be a platform that provides a penetration testing company a solution for tracking penetration test results and system vulnerabilities for given clients while also enabling their clients access the system to view the related findings. Obviously in both cases the data integrated in such a system is sensitive and the system must provide the necessary protection of data in terms of security and privacy.
In this post I will hold on to the later example. That is, we will have a penetration testing company that needs a software solution in form of a web application, that stores client information and case findings, while allowing a penetration tester as well as an end client (who hires the penetration tester) to access and use the system. For instance, a penetration tester can login and view findings data or create new cases, while an end client can view cases and findings data related to his account.
The architecture is based on Model B of the generic concept of the TMF (Technology and Methodology Platform for Networking in Medical Research). The functions are available through a RESTful interface.
In order to ensure the End Client's privacy, Identification Data (IDAT) and Findings Data (VDAT) will be stored encrypted on separate servers associated with the help of a Trusted Third Party (TTP), which we call Pseudonymization Service (PSNS). This design concept has the benefit that all three servers (IDAT, PSNS and VDAT) have to be compromised in order to trace any found vulnerabilities back to the end client. In addtion to IDAT, PSNS and VDAT we will also have a fourth server called APP that will handle application data and the GUI. APP, IDAT, PSNS and VDAT will be referred as modules from now on.
For a more in depth description of the architecture please read:
- Distributed system for data security and privacy – Part 1
- Distributed system for data security and privacy – Part 2
- 1. Required Software
- 2. Setup Database Server
- 3. Encrypt database password using Jasypt
- 4. Create a self-signed certificate
- 5. Configure Tomcat SSL Connector
- 6. Import project with IntelliJ IDEA
- 7. Setup IDE Tomcat
- 8. Run Tomcat from the IDE
- 9. Project Structure
- 10. Project specific conventions
- 11. Important Configuration Properties
- 12. Tomcat Ports
- 13. Test Users
Download and install Java 1.8+ JDK.
Download and install Apache Tomcat 8.
Download and install Gradle.
Download and install an IDE. For example IntelliJ IDEA.
Download and install Jasypt.
Download and install a working MySQL database server. You can use Oracle MySQL or MariaDB.
- Install your MySQL server.
- Start the server:
sudo mysql.server start - Set the MySQL root password:
mysqladmin -u root password 'mysql' - Create a new database for each module and name them
app,idat,psns,vdatrespectively.
This step is only for informational purposes since the database and database encryption password are already configured in encrypted form in the db.properties property file.
Because we will place this password in a property file so the application can access it, we need to encrypt it first using Jasypt:
- Open your Terminal and change to your Jasypt installation directory.:
cd /Users/lucas/opt/jasypt-1.9.2/bin - Run:
encrypt.sh input="mysql" password="master password" algorithm="PBEWITHMD5ANDDES". The output should look as follows:
----ENVIRONMENT----------------- Runtime: Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 25.25-b02 ----ARGUMENTS------------------- algorithm: PBEWITHMD5ANDDES input: mysql password: master password ----OUTPUT---------------------- d22XcE0UgwoLdUyAYcLuGA==
The encrypted password is d22XcE0UgwoLdUyAYcLuGA==.
To decrypt the password you can run: decrypt.sh input="d22XcE0UgwoLdUyAYcLuGA==" password="master password" algorithm="PBEWITHMD5ANDDES".
Although this is the password for the database connection, we will use the same password for the database encryption during development for simplicity reasons. We can repeat this step for arbitrary configuration properties like URLs, ports, etc, later.
In this step we will generate a keystore with self-signed certificate in it. So go ahead and fire up your Terminal and run the following command:
keytool -genkey -alias keyAlias -storetype PKCS12 -keyalg RSA -keysize 2048 -keystore txlKeystore.p12 -validity 3650
Executing this command will ask you a few identity questions:
Enter keystore password: 123456 Re-enter new password: 123456 What is your first and last name? [Unknown]: 127.0.0.1 What is the name of your organizational unit? [Unknown]: txl What is the name of your organization? [Unknown]: txl What is the name of your City or Locality? [Unknown]: txl What is the name of your State or Province? [Unknown]: txl What is the two-letter country code for this unit? [Unknown]: txl Is CN=127.0.0.1, OU=txl, O=txl, L=txl, ST=txl, C=txl correct? [no]: yes
Issue the `list` command to ensure that the keystore contains a certificate:
keytool -list -keystore txlKeystore.p12 -storetype PKCS12
The output should look similar to this:
> ```
> Enter keystore password: 123456
>
> Keystore type: PKCS12
> Keystore provider: SunJSSE
>
> Your keystore contains 1 entry
>
> keyalias, Aug 8, 2015, PrivateKeyEntry,
> Certificate fingerprint (SHA1): 4F:53:4E:E5:12:70:F7:74:20:1B:56:1E:44:85:D0:23:2F:21:38:C3
Place the txlKeystore.p12 file in a directory where you can reference it later. Example: /Users/lucas/txlKeystore.p12.
In this step we will configure our Tomcat server to support HTTPS. For that we need to configure an SSL connector. Open up lucas/opt/apache-tomcat-8.0.15/conf/server.xml and add the following line:
<Connector SSLEnabled="true" keystoreFile="/Users/lucas/txlKeystore.p12" keystoreType="PKCS12" keystorePass="123456" port="8443" scheme="https" secure="true" sslProtocol="TLS"/> above the line where it says <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />.
IMPORTANT: Since we will deploy our modules on 4 different Tomcats, we need to adjust the configuration accordingly. This can be done either by copying an existing Tomcat installation and adjusting the Connector configuration in server.xml to enable the port as defined in 12. Tomcat Ports, or use one existing Tomcat installation and point Eclipse to use a different server.xml for each Run Configuration. The later is not possible in IntelliJ.
Start IntelliJ IDEA. A window will appear with options. Select Import Project.
| Screenshot 1.1: IntelliJ Import |
|---|
 |
Browse to the module directory and select the root directory prototype.
IMPORTANT: not the repository directory! If you repository is called distributed-system-tmf-model-b, then you should select distributed-system-tmf-model-b/prototype.
| Screenshot 1.2: Select Project |
|---|
 |
In the next view select Gradle and click Next
| Screenshot 1.3: Select Build Tool |
|---|
 |
Make sure the setup looks like Screenshot 1.4 and click Finish. If no JVM is available do the following:
- Close the window and go to the window of Screenshot 1.1.
- Navigate to
Configure -> Project Defaults -> Project Structure -> Platform Settings. - Click on the
+button and selectJDKto add a JDK.
| Screenshot 1.4: Gradle Distribution |
|---|
 |
Wait until Gradle fetches all dependencies and builds the project.
| Screenshot 1.5: Finish & Build |
|---|
 |
When done, the project structure should look as follows:
| Screenshot 1.6: Project Structure |
|---|
 |
Next we need to setup a global Tomcat application server so we can configure our Tomcat Run Configurations for each module later.
Open up IntelliJ preferences (⌘+,) and navigate to the section Application Servers.
| Screenshot 2.1: Application Servers settings |
|---|
 |
Click the + button to add a new application server and select Tomcat Server.
| Screenshot 2.2: Add Tomcat Server |
|---|
 |
Select your Tomcat Home directory (installation directory) and click OK.
| Screenshot 2.3: Tomcat Home |
|---|
 |
The setup should look similar to Screenshot 2.4.
IMPORTANT FOR INTELLIJ USERS: Since we will be using 4 Tomcat Run Configurations, each of which will have a different server.xml configuration, it is better if you setup 4 Tomcats, with each IntelliJ Tomcat pointing to a different Tomcat Home installation directory. For Eclipse users the replication of Tomcat installations is not necessary, since Eclipse allows the specification of a different server.xml file for each Run Configuration (Project Explorer -> Servers -> Tomcat XY).
| Screenshot 2.4: Tomcat setup |
|---|
 |
| All modules Tomcat |
Next we need to setup a Run Configuration. Although this guide is for the APP module, the following steps should be repeated for all other modules since we want to simulate a live scenario where every module runs on a separate Tomcat.
Go to IntelliJ run configurations: Run -> Edit Configurations....
| Screenshot 2.5: Run Configurations |
|---|
 |
Click on the + to add a new run configuration and select Tomcat Server -> Local.
| Screenshot 2.6: Select Tomcat Server |
|---|
 |
Name it APP as this will be the run configuration for APP.
HOT DEPLOYMENT: If you want to enable hot deployment, you can choose Update classes and resources under On 'Update' action and under On frame deactivation.
IMPORANT: Make sure the ports used for every module are different! See 12. Tomcat Ports.
IMPORTANT FOR INTELLIJ USERS: If you created 4 different Tomcats earlier, make sure to select the right Tomcat under Application server.
| Screenshot 2.7: Run Configuration APP |
|---|
 |
| See also: IDAT - PSNS - VDAT |
Switch to the Deployment tab and click on the + to add a new Artifact that should be deployed. Select app.war or the app.war (exploded) for hot deployment. Click OK.
| Screenshot 2.8: Select Artifacts to deploy |
|---|
 |
The setup should look as Screenshot 2.9.
IMPORANT: Make sure that the Application context is set to module-name! For APP the context must be /app. For IDAT the context must be /idat. For PSNS the context must be /psns. For VDAT the context must be /vdat.
| Screenshot 2.9: Artifacts to deploy |
|---|
 |
Switch to the tab Startup/Connection and add a new environment variable named PASSWORD_ENV_VARIABLE with value master password. This variable will be passed to the application on startup and its value will be used to decrypt any encrypted properties later on. This is the same master password as used in step Encrypt database password using Jasypt. Click OK.
| Screenshot 2.10: Environment Variable |
|---|
 |
Also setup the PASSWORD_ENV_VARIABLE for Debug modus.
| Screenshot 2.11: Environment Variable Debug |
|---|
 |
In this step we will start our Tomcat Servers and test our freshly deployed application.
Hover over or click the icon on the bottom left corner in the IntelliJ project window. Select Application Servers. This will open up a view with all your configured application server run configurations.
| Screenshot 3.1: Show Application Servers Run Configurations |
|---|
 |
Select APP.
| Screenshot 3.2: IDE Start Tomcat |
|---|
 |
Click on the green play button (
) to start the Tomcat server. The application should have deployed without any problems
and the console output should look similar to Screenshot 3.3.
| Screenshot 3.3: IDE Start Tomcat APP |
|---|
 |
Next, start the IDAT Tomcat. The application should have deployed without any problems
and the console output should look similar to Screenshot 3.4.
| Screenshot 3.4: IDE Start Tomcat IDAT |
|---|
 |
Next open up your web browser and connect to https://localhost:8443/app. You will see something similar to Screenshot 3.5.
| Screenshot 3.5: Open application in browser |
|---|
 |
Examining the certificate details you will notice that those are the details entered in step Create a self-signed certificate.
| Screenshot 3.6: Certificate details |
|---|
 |
Clicking on Continue will lead you to the login page.
| Screenshot 3.7: Login Page |
|---|
 |
Repeat this step for all modules and accept the certificate, if necessary. This is required because the application requests resources from different servers and the cerificate has to be accepted first.
- https://localhost:8443/app/
- https://localhost:8444/idat
- https://localhost:8445/psns
- https://localhost:8446/vdat
IMPORTANT: Notice the different port!
Once done return to the login page and click on login and login using the username = admin and password = admin. This will trigger a login on the APP, IDAT, VDAT and PSNS server (Cross Origin) and redirect you to the main page.
| Screenshot 3.8: Main Page |
|---|
 |
You can navigate to Cases -> New and create a new case. When finished, the client identification data (Client Profile) will have been saved into IDAT while case relevant data will have been saved into VDAT.
| Screenshot 3.9: Create New Case |
|---|
 |
You can navigate to Cases -> Overview. This will fetch client information from IDAT as well as case data from VDAT.
| Screenshot 3.10: Fetch Case |
|---|
 |
The Java Projects for each module will be Gradle projects and multiple project builds. That is, they will consist of different subprojects. The project structure for a single module (module-name) is listed below:
| Project Name | Description |
|---|---|
| module-name (app, idat, psns, vdat) | Parent project that connects subprojects. No source code in here. |
| module-name-presentation | Public API of module. It is a collection of Spring Controllers that handle HTTP requests. It also holds the Spring configuration files. In the APP module this subproject also holds the views. |
| module-name-service | Service Spring bean implementations go in here. A Service bean does the business logic of the module. It is usually called by the Controllers in the module-presentation project. |
| module-name-domain | Domain objects go in here. These are usually objects that are handled by Service beans and will be persisted to the database. Example: FrameCondition. |
| module-name-api | Interfaces for Service beans go in here. module-api consists only of interfaces that are implemented in module-service and Exceptions that are used in module-service |
| module-name-dao | Interfaces for DAO. module-dao consists only of interfaces that are implemented in module-dummy or module-dao-hibernate |
| module-name-dao-dummy | Dummy implementation of the DAO interfaces defined in module-dao. DAOs are usually called by Service beans. DAOs make use of module-domain objects. A DAO dummy does not persist any data to the database. It creates/reads dummy data and simulates data persistence/read. |
| module-name-dao-hibernate | Hibernate implementation of the DAO interfaces defined in module-dao. DAOs are usually called by Service beans. DAOs make use of module-domain objects. Any database configuration, that is not a Spring configuration, goes in here. Such a configuration can be a Hibernate configuration (hibernate.cfg.xml), a Hibernate mapping configuration (FrameCondition.hbm.xml), Jasypt configuration (jasyptHibernateTypes.hbm.xml), etc. Any Spring related configuration, such as sessionFactory, transactionManager, etc goes into the Spring configuration files located in the module-presentation project. Hibernate DAOs persist data to the database using the Hibernate ORM framework. |
In addition to the above sub projects we also have a project called common which follows the same inner structure (api, dao, etc) as all the other modules. This project holds functionality that is common to all modules, such as user management.
| Project Name | Description |
|---|---|
| common | This project holds functionality that is common to all modules, such as user management. |
This is to avoid confusing application classes and interfaces from framework (Spring, Hibernate, etc) components. We will use the prefix TXL for all classes.
Do this:
public class TXLUser {
}Not this:
public class User {
}| Name | Scope | Value | Encrypted Value | Description |
|---|---|---|---|---|
db.password |
Database Connection Password | mysql |
d22XcE0UgwoLdUyAYcLuGA== |
Use ENC(d22XcE0UgwoLdUyAYcLuGA==) when storing in property file |
db.encryptionPassword |
Database Encryption Password | mysql |
d22XcE0UgwoLdUyAYcLuGA== |
Use ENC(d22XcE0UgwoLdUyAYcLuGA==) when storing in property file |
PASSWORD_ENV_VARIABLE |
Properties Decryption Password | master password |
Setup as environment variable in Tomcat configuration |
| Module | HTTP Port | HTTPS Port | JMX Port | Application Context | URL |
|---|---|---|---|---|---|
| APP | 8080 | 8443 | 1099 | /app | https://127.0.0.1:8443/app or https://localhost:8443/app |
| IDAT | 8081 | 8444 | 1100 | /idat | https://127.0.0.1:8444/idat or https://localhost:8444/idat |
| PSNS | 8082 | 8445 | 1101 | /psns | https://127.0.0.1:8445/psns or https://localhost:8445/psns |
| VDAT | 8083 | 8446 | 1102 | /vdat | https://127.0.0.1:8446/vdat or https://localhost:8446/vdat |
Below is a list of development users that can be used to test the different functionalities of the system.
| Username | Password | Type | Role | Comment | Sites |
|---|---|---|---|---|---|
| admin | admin | PENTESTER | ADMIN | SiteA, SiteB | |
| puser1 | puser1 | PENTESTER | USER | SiteA, SiteB | |
| puser2 | puser2 | PENTESTER | USER | SiteA, SiteB | |
| ecuser1 | ecuser1 | END CLIENT | USER | SiteA, SiteB | |
| ecuser1 | ecuser1 | END CLIENT | USER | SiteA, SiteB |