Mingtao's Java World

Thursday, 2 May 2019

MySQL Cheatsheet

Open MySQL command line

mysql -u root -p

Show all databases

show databases;

Create new database

CREATE DATABASE dl_edeposit;

Show all users

SELECT host, user, password FROM mysql.user;

Create new user

CREATE USER 'dl_edeposit'@'localhost' IDENTIFIED BY 'password';

Grant privilege to a user on a table

GRANT ALL PRIVILEGES ON dl_edeposit.* TO 'dl_edeposit'@'localhost';

Create SSH public/private key

The aim is to be able to connect to (ssh) a remove server without a password.

you@local$ssh-keygen -t rsa -b 4096 -C [your username (e.g. msun]

When you're prompted to "Enter a file in which to save the key," press Enter. This accepts the default file location.
Enter your password twice.

Enter passphrase (empty for no passphrase): [Type a passphrase]
Enter same passphrase again: [Type passphrase again]

Copy the public key to the remote server

you@local$ cat ~/.ssh/id_rsa.pub | ssh msun@<remove server> 'cat - >> ~/.ssh/authorized_keys'
you@remote's password:

Store your password to the keychain. This ensures you don't need to type password in other terminal session

ssh-add -K ~/.ssh/id_rsa

Reference:

http://mah.everybody.org/docs/ssh#upload-public-key
https://help.github.com/articles/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent/

Tuesday, 30 April 2019

Configure Sonarqube external database

Sonarqube documentation says the in-memory database should be used for evaluation purpose. For production purpose, a more scalable solution is needed -- a standalone database.

Despite the warning on the documentation against MySQL, I decided to give it a go anyway because

1. I have already installed a MySQL server locally, and
2. I was just trying to test a solution to a problem

After having created the database scheme, the user, and granted the privileges, I removed the old sonarqube container and attempted to start the new one with the following command.

docker run --name sonarqube\
    -p 9000:9000 \
    -e sonar.jdbc.username=sonarqube \
    -e sonar.jdbc.password=<password> \
    -e sonar.jdbc.url="jdbc:mysql://localhost:3306/sonarqube?useUnicode=true&characterEncoding=utf8" \
    sonarqube

The I got this error.

019.05.01 06:15:52 INFO  web[][o.sonar.db.Database] Create JDBC data source for jdbc:mysql://localhost:3306/sonarqube?useUnicode=true&characterEncoding=utf8
2019.05.01 06:15:53 ERROR web[][o.s.s.p.Platform] Web server startup failed
java.lang.IllegalStateException: Can not connect to database. Please check connectivity and settings (see the properties prefixed by 'sonar.jdbc.').
 at org.sonar.db.DefaultDatabase.checkConnection(DefaultDatabase.java:119)
 at org.sonar.db.DefaultDatabase.start(DefaultDatabase.java:85)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.picocontainer.lifecycle.ReflectionLifecycleStrategy.invokeMethod(ReflectionLifecycleStrategy.java:110)
 at org.picocontainer.lifecycle.ReflectionLifecycleStrategy.start(ReflectionLifecycleStrategy.java:89)
 at org.picocontainer.injectors.AbstractInjectionFactory$LifecycleAdapter.start(AbstractInjectionFactory.java:84)
 at org.picocontainer.behaviors.AbstractBehavior.start(AbstractBehavior.java:169)
 at org.picocontainer.behaviors.Stored$RealComponentLifecycle.start(Stored.java:132)
 at org.picocontainer.behaviors.Stored.start(Stored.java:110)
 at org.picocontainer.DefaultPicoContainer.potentiallyStartAdapter(DefaultPicoContainer.java:1016)
 at org.picocontainer.DefaultPicoContainer.startAdapters(DefaultPicoContainer.java:1009)
 at org.picocontainer.DefaultPicoContainer.start(DefaultPicoContainer.java:767)
 at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:135)
 at org.sonar.server.platform.platformlevel.PlatformLevel.start(PlatformLevel.java:90)
 at org.sonar.server.platform.platformlevel.PlatformLevel1.start(PlatformLevel1.java:154)
 at org.sonar.server.platform.Platform.start(Platform.java:211)
 at org.sonar.server.platform.Platform.startLevel1Container(Platform.java:170)
 at org.sonar.server.platform.Platform.init(Platform.java:86)
 at org.sonar.server.platform.web.PlatformServletContextListener.contextInitialized(PlatformServletContextListener.java:45)
 at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4817)
 at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5283)
 at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
 at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1423)
 at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1413)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748)
Caused by: java.sql.SQLException: Cannot create PoolableConnectionFactory (Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.)
 at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2385)
 at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:2110)
 at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1563)
 at org.sonar.db.profiling.NullConnectionInterceptor.getConnection(NullConnectionInterceptor.java:31)
 at org.sonar.db.profiling.ProfiledDataSource.getConnection(ProfiledDataSource.java:317)
 at org.sonar.db.DefaultDatabase.checkConnection(DefaultDatabase.java:116)
 ... 30 common frames omitted
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

Guess what, since we have installed sonarqube using docker, and the sonarqube server actually sits inside a container. The localhost IP address is not exposed to the container. The fix is to replace localhost with host.docker.internal:

docker run --name sonarqube\
    -p 9000:9000 \
    -e sonar.jdbc.username=sonarqube \
    -e sonar.jdbc.password=password \
    -e sonar.jdbc.url="jdbc:mysql://host.docker.internal:3306/sonarqube?useUnicode=true&characterEncoding=utf8" \
    sonarqube

Sonarqube coverage is always 0? Jacoco quick start

When I was playing around with Sonarqube, I noticed the the coverage is always 0.0%, although it does display the unit tests have all passed.

Then it shouldn't be 0.0%, should it?

At first, I tried specifying the report path without any luck.

-Dsonar.junit.reportsPath=target/surefire-reports

Then I read a thread in which a guy had the same issue and he claimed to solve it by abandoning the default in-memory database in favor of an external database.

I tried that also to no avail.

At last, I came upon the official documentation regarding the unit test coverage, and it clearly states that

SonarSource analyzers do not run your tests or generate reports. They only import pre-generated reports. Below you'll find language- and tool-specific analysis parameters for importing coverage and execution reports.

Wow, if only I had read it earlier.

For Java language, the popular tool to generate coverage report is Jacoco.

Thanks to this wonderful tutorial, setting up Jacoco is made really easy.

All you need to do is copy the xml below under the build element of the pom.xml

<plugins>
    <plugin>
        <groupId>org.jacoco</groupId>
        <artifactId>jacoco-maven-plugin</artifactId>
        <version>0.8.3</version>
        <executions>
            <!-- Prepares the property pointing to the JaCoCo runtime agent 
                which is passed as VM argument when Maven the Surefire plugin is executed. -->
            <execution>
                <id>pre-unit-test</id>
                <goals>
                    <goal>prepare-agent</goal>
                </goals>
                <configuration>
                    <!-- Sets the path to the file which contains the execution 
                        data. -->
                    <destFile>${project.build.directory}/coverage-reports/jacoco-ut.exec
                    </destFile>
                    <!-- Sets the name of the property containing the settings 
                        for JaCoCo runtime agent. -->
                    <propertyName>surefireArgLine</propertyName>
                </configuration>
            </execution>
            <!-- Ensures that the code coverage report for unit tests is 
                created after unit tests have been run. -->
            <execution>
                <id>post-unit-test</id>
                <phase>test</phase>
                <goals>
                    <goal>report</goal>
                </goals>
                <configuration>
                    <!-- Sets the path to the file which contains the execution 
                        data. -->
                    <dataFile>${project.build.directory}/coverage-reports/jacoco-ut.exec
                    </dataFile>
                    <!-- Sets the output directory for the code coverage 
                        report. -->
                    <outputDirectory>${project.reporting.outputDirectory}/jacoco-ut
                    </outputDirectory>
                </configuration>
            </execution>
        </executions>
    </plugin>
    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>3.0.0-M3</version>
        <configuration>
            <!-- Sets the VM argument line used when unit tests are run. -->
            <argLine>${surefireArgLine}</argLine>
            <!-- Skips unit tests if the value of skip.unit.tests property 
                is true -->
            <skipTests>${skip.unit.tests}</skipTests>
        </configuration>
    </plugin>
</plugins>

The maven command to generate the report is as simple as:

mvn clean test

The maven command to run Sonarqube with Jacoco is:

mvn clean package sonar:sonar -Dsonar.projectKey=<key> -Dsonar.host.url=http://localhost:9000 -Dsonar.login=<token> -Dsonar.junit.reportsPath=target/surefire-reports -Dsonar.coverage.jacoco.xmlReportPaths=target/site/jacoco-ut/jacoco.xml

Monday, 29 April 2019

SonarQube quick start with Docker

I strongly recommend using SonarQube to scan your code. Here is how to set it up locally.

1. Download and install Docker. (Docker is a container that allows developers to package up an application with all the parts it needs)
https://docs.docker.com/install/
2. Install SonarQube in docker and start the server
https://hub.docker.com/_/sonarqube/

docker run -d --name sonarqube -p 9000:9000 sonarqube

3. Go to http://localhost:9000, login as admin (username: admin, password: admin)
4. Click 'create new project', follow the wizard, and note down the token (e.g. 15be81b4036cc5034363a43af4aa19773ea87242)
5. Run the maven command

mvn clean package sonar:sonar -Dsonar.projectKey=test -Dsonar.host.url=http://localhost:9000 -Dsonar.login=15be81b4036cc5034363a43af4aa19773ea87242

6. View the result at http://localhost:9000

Don't call non-final methods from constructors

You would expect the following program outputs 10.

public class ConstuctorTest {
    public static void main(String args[]) {
        Derived d = new Derived();
        System.out.println("From main() d.value() returns " + d.value());
    }
}

class Base {
    private int val;

    public Base() {
        val = lookup();
    }

    public int lookup() {
        return 5;
    }

    public int value() {
        return val;
    }
}

class Derived extends Base {
    
    private int num = 10;

    public Derived() {
        super();
    }
    
    public int lookup() {
        return num;
    }
}

The actual outcome, surprisingly, is:

From main() d.value() returns 0

Have a look at the order of statement execution.

public class ConstructorTest {
    public static void main(String args[]) {
        Derived d = new Derived();
        System.out.println("From main() d.value() returns " + d.value());
    }
}

class Base {
    private int val;

    public Base() {
        System.out.println("Base constructor"); // 1
        val = lookup();
    }

    public int lookup() {
        return 5;
    }

    public int value() {
        return val;
    }
}

class Derived extends Base {
    
    private int num = generateNum();

    public Derived() {
        super();
        System.out.println("Derived constructor num = " + num); // 4
    }
    
    private int generateNum() {
        System.out.println("Generate num"); // 3
        return 10;
    }
    
    public int lookup() {
        System.out.println("Look up num: "+num); // 2
        return num;
    }
}

Output:

Base constructor
Look up num: 0
Generate num
Derived constructor num = 10
From main() d.value() returns 0

It turns out the assignment of instance variable happens after the calling of the constructor of the super class, and before the very first statement of the constructor of this class.

The lesson is if you want to call an instance method in the constructor, you should declare the method as final.