Monday, 9 September 2019

Python Sonarqube with unit test coverage

Prerequisite

Sonarqube is installed (Create a new project named 'shootout')
Sonar scanner is installed
Anaconda is installed

Install nose and coverage

pip install nose
pip install coverage

Python Project structure

main.py
sample/
    hello.py
testcase/
    test.py

Sonar property file

create sonar-project.properties under the root folder

Contents of the file

sonar.projectKey=shootout #corresponds to the project name in Sonarqube
sonar.projectName=Penalty shootout
sonar.projectVersion=1.0
sonar.language=py
sonar.python.xunit.reportPath=nosetests.xml
sonar.python.coverage.reportPaths=coverage.xml
sonar.python.xunit.skipDetails=true
sonar.coverage.exclusions=testcase/**/* 

Activate the Anaconda virtual environment

Note: It appears that nose can only run in the virtual environment, hence the need of the activation.

conda init

Generate nosetests.xml and coverage.xml

coverage erase 
nosetests --with-coverage --with-xunit
coverage xml

Analyze the project

sonar-scanner

View the result

http://localhost:9000

Troubleshooting

https://github.com/sunmingtao/sample-code/issues/11
https://github.com/sunmingtao/sample-code/issues/12
https://github.com/sunmingtao/sample-code/issues/13
https://github.com/sunmingtao/sample-code/issues/14
https://github.com/sunmingtao/sample-code/issues/15
https://github.com/sunmingtao/sample-code/issues/16

Sunday, 1 September 2019

JMockit incompatible with JDK 10/11

Running Unit tests under JDK 10/11 produces error below. It was OK under JDK 8.

[ERROR] Errors: 
[ERROR]   BusinessRepositoryTest>AbstractRepositoryTest.setUp:23 » NoClassDefFound Could...
[ERROR]   BusinessRepositoryTest>AbstractRepositoryTest.setUp:23 » ExceptionInInitializer
[ERROR]   BusinessUserRepositoryTest>AbstractRepositoryTest.setUp:23 » NoClassDefFound C...
[ERROR]   BusinessUserRepositoryTest>AbstractRepositoryTest.setUp:23 » NoClassDefFound C...
[ERROR]   TransactionRepositoryTest>AbstractRepositoryTest.setUp:23 » NoClassDefFound Co...
[ERROR]   TransactionRepositoryTest>AbstractRepositoryTest.setUp:23 » NoClassDefFound Co...
[ERROR]   TransactionRepositoryTest>AbstractRepositoryTest.setUp:23 » NoClassDefFound Co...
[ERROR]   BusinessCustomerLinkServiceTest>AbstractRepositoryTest.setUp:23 » NoClassDefFound
[ERROR]   BusinessServiceTest.save:30 » NullPointer
[INFO] 
[ERROR] Tests run: 9, Failures: 0, Errors: 9, Skipped: 0

The contents of AbstractRepositoryTest is nothing but a Mockup of a public static method

@BeforeEach
void setUp() throws Exception {
    new MockUp<CafamilyUtils>() {
        @mockit.Mock
        public String getLoggedInUserID() {
            return "system";
        }
    };
}

The version of jmockit is too old (1.24) in pom.xml. It needs to be upgraded to a much newer version (1.47). Besides JMockit also requires the -javaagent JVM initialization parameter to be used, according to
https://jmockit.github.io/tutorial/Introduction.html#runningTests

<dependencies>
   <dependency>
      <groupId>org.jmockit</groupId>
      <artifactId>jmockit</artifactId>
      <version>${jmockit.version}</version>
      <scope>test</scope>
   </dependency>
</dependencies>


<plugins>
   <plugin>
      <artifactId>maven-surefire-plugin</artifactId>
      <version>2.22.2</version> <!-- or some other version -->
      <configuration>
         <argLine>
            -javaagent:${settings.localRepository}/org/jmockit/jmockit/${jmockit.version}/jmockit-${jmockit.version}.jar
         </argLine>
      </configuration>
   </plugin>
</plugins>

Sunday, 25 August 2019

Static method MockUp effect is global

Use mockit.MockUp.MockUp() to mock public static methods. One caveat is that the mocked behaviour applies to other test methods as well. (test2() prints "abc", which is surprising, isn't it?)

@ExtendWith(value = {MockitoExtension.class})
@RunWith(JUnitPlatform.class)
class MyTest {
    
    @Test
    void test1() throws IOException {
        new MockUp<System>() {
            @mockit.Mock
            public String getProperty(String key){
                return "abc";
            }
            
        };
        
        assertThat(System.getProperty("a"), is("abc"));
    }
    
    @Test
    void test2() throws IOException {
        System.out.println(System.getProperty("a")); //Prints abc
    }
    

}

Junit 5 temp directory quick start

The tempDirectory extension makes it quite handy to test file reading/writing functions.

Maven dependency

<dependency>
   <groupId>org.junit-pioneer</groupId>
   <artifactId>junit-pioneer</artifactId>
   <version>0.1.2</version>
   <scope>test</scope>
</dependency>

Java

@Test
@ExtendWith(TempDirectory.class)
void withProperties(@TempDir Path tempDir) throws IOException {
    Path path = tempDir.resolve("env.properties");
    String properties = "a=b";
    Files.write(path, properties.getBytes());
    Map<String, Object> props = EnvPropertiesLoader.loadEnvProperties();
    assertThat(props.get("a"), is("b"));
}

Thursday, 22 August 2019

Double-checked locking should not be used

Extracted from sonarqube

Double-checked locking is the practice of checking a lazy-initialized object's state both before and after a synchronized block is entered to determine whether or not to initialize the object.
It does not work reliably in a platform-independent manner without additional synchronization for mutable instances of anything other than float or int. Using double-checked locking for the lazy initialization of any other type of primitive or mutable object risks a second thread using an uninitialized or partially initialized member while the first thread is still creating it, and crashing the program.

There are multiple ways to fix this. The simplest one is to simply not use double checked locking at all, and synchronize the whole method instead. With early versions of the JVM, synchronizing the whole method was generally advised against for performance reasons. But synchronized performance has improved a lot in newer JVMs, so this is now a preferred solution. If you prefer to avoid using synchronized altogether, you can use an inner static class to hold the reference instead. Inner static classes are guaranteed to load lazily.

Noncompliant Code Example

@NotThreadSafe
public class DoubleCheckedLocking {
    private static Resource resource;

    public static Resource getInstance() {
        if (resource == null) {
            synchronized (DoubleCheckedLocking.class) {
                if (resource == null)
                    resource = new Resource();
            }
        }
        return resource;
    }

    static class Resource {

    }
}

Compliant Solution

@ThreadSafe
public class SafeLazyInitialization {
    private static Resource resource;

    public synchronized static Resource getInstance() {
        if (resource == null)
            resource = new Resource();
        return resource;
    }

    static class Resource {
    }
}
With inner static holder:
@ThreadSafe
public class ResourceFactory {
    private static class ResourceHolder {
        public static Resource resource = new Resource(); // This will be lazily initialised
    }

    public static Resource getResource() {
        return ResourceFactory.ResourceHolder.resource;
    }

    static class Resource {
    }
}
Using "volatile":

class ResourceFactory {
  private volatile Resource resource;

  public Resource getResource() {
    Resource localResource = resource;
    if (localResource == null) {
      synchronized (this) {
        localResource = resource;
        if (localResource == null) {
          resource = localResource = new Resource();
        }
      }
    }
    return localResource;
  }

  static class Resource {
  }
}

Wednesday, 21 August 2019

Swift non-optional, optional (?), and optional(!)


class Country {
    let name: String
    var capitalCity: City!
    init(name: String, capitalName: String) {
        self.name = name
        self.capitalCity = City(name: capitalName, country: self)
    }
    deinit {
        print ("country deinit")
    }
}

class City {
    let name: String
    unowned let country: Country
    init(name: String, country: Country) {
        self.name = name
        self.country = country
    }
    deinit {
        print ("city deinit")
    }
}

var country: Country = Country(name: "Canada", capitalName: "Ottawa")
print ("country capital city is \(country.capitalCity.name)")

Why the exclamation mark is needed after City?

Without making it the variable optional, the code won't compile, because the country instance won't be considered fully initialised and therefore cannot be passed as an argument to initiate the city instance (country: self).

Can exclamation mark be replaced by question mark?

Yes, it can. But to access this property, it needs to be unwrapped. So country.capitalCity.name is to be replaced by country.capitalCity!.name. If we know for sure a property will never be null, it's better to be defined as City! rather than City?.

Swift Automatic reference counting (Weak variable)

Weak reference


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
class Person{
    var name: String
    var fruit: Fruit?
    
    init(name: String) {
        self.name = name
        print ("Person init")
    }
    
    deinit {
        print ("Person denit")
    }
}

class Fruit{
    var name: String
    init(name: String) {
        self.name = name
        print ("Fruit init")
    }
    
    deinit {
        print ("Fruit denit")
    }
}

var person: Person? = Person(name: "john")
person!.fruit = Fruit(name: "apple")
person = nil //Prints "Person denit", "Fruit denit"

After line 28, the fruit object has one strong reference on it (the property person.fruit), and the person object also has one strong reference on it (the variable person).

At line 29, two things happen.

1. The person object has no reference on it and thus gets deallocated. It prints "Person deinit"
2. Since the person object is gone, the fruit object has no reference on it, and thus gets deallocated. It prints "Fruit deinit"


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
class Person{
    var name: String
    var fruit: Fruit?
    
    init(name: String) {
        self.name = name
        print ("Person init")
    }
    
    deinit {
        print ("Person denit")
    }
}

class Fruit{
    var name: String
    var person: Person?
    
    init(name: String) {
        self.name = name
        print ("Fruit init")
    }
    
    deinit {
        print ("Fruit denit")
    }
}

var person: Person? = Person(name: "john")
var fruit: Fruit? = Fruit(name: "apple")
person!.fruit = fruit
fruit!.person = person
person = nil
fruit = nil //Nothing is deinitilised. 

After line 32, the fruit object has two strong references on it (the variable fruit, and the property person.fruit). Similarly the person object has two strong references on it (the variable person, and the property fruit.person)

After line 34, both the fruit objects and the person objects have one strong reference on it (the property person.fruit and fruit.person respectively). Hence nothing is deinitilised.


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
class Person{
    var name: String
    weak var fruit: Fruit?
    
    init(name: String) {
        self.name = name
        print ("Person init")
    }
    
    deinit {
        print ("Person deinit")
    }
}

class Fruit{
    var name: String
    var person: Person?
    
    init(name: String) {
        self.name = name
        print ("Fruit init")
    }
    
    deinit {
        print ("Fruit deinit")
    }
}

var person: Person? = Person(name: "john")
var fruit: Fruit? = Fruit(name: "apple")
person!.fruit = fruit
fruit!.person = person
fruit = nil //Prints "Fruit deinit"

After line 32, the fruit object has one strong reference (the variable fruit) and one weak reference (the property person.fruit)

At line 33, the strong reference is broken, and there is only one weak reference left, which can't stop the object being deinitialised.



 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
class Person{
    var name: String
    weak var fruit: Fruit?
    
    init(name: String) {
        self.name = name
        print ("Person init")
    }
    
    deinit {
        print ("Person deinit")
    }
}

class Fruit{
    var name: String
    var person: Person?
    
    init(name: String) {
        self.name = name
        print ("Fruit init")
    }
    
    deinit {
        print ("Fruit deinit")
    }
}

var person: Person? = Person(name: "john")
var fruit: Fruit? = Fruit(name: "apple")
person!.fruit = fruit
fruit!.person = person
person = nil
fruit = nil //Prints "Fruit deinit, Person deinit"

After line 32, the fruit object has one strong reference (the variable fruit) and one weak reference (the property person.fruit). The person object two strong references (the variable person and the property fruit.person)

At line 33, one strong reference on the person object is broken, but since there is another strong reference, object person can't be deallocated.

At line 34, the strong reference on the fruit object is broken, the fruit object is deallocated, thus breaking the other strong reference on the person object. Hence the person is deallocated.

Unowned reference


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
class Person{
    var name: String
    var fruit: Fruit?
    
    init(_ name: String) {
        self.name = name
        print ("Person init")
    }
    
    deinit {
        print ("Person deinit")
    }
}

class Fruit{
    var name: String
    unowned var person: Person
    
    init(_ name: String, person: Person) {
        self.name = name
        self.person = person
        print ("Fruit init")
    }
    
    deinit {
        print ("Fruit deinit")
    }
}

var person: Person? = Person("john")
person!.fruit = Fruit("apple", person: person!)
person = nil //Prints "Person deinit, Fruit deinit"

After line 31, the person object has one strong reference (var person), and one unowned reference (property fruit.person).

At line 32, the strong reference on the person object is broken, and the person object is deallocated.

Note unowned cannot be used on optional type. (i.e. replacing unowned by weak in the above code causes compilation error)