Handy Shell Commands for Beginners

Via Wikimedia Commons

Check if a file exists

if [ ! -f "/home/pi/test.sh" ]; then
echo "file exists"
fi

Check if a directory exists

if [ -d "/home/pi/test" ]; then
echo "test dir exists"
fi

Install Apt-Get Package

# update pip
# check if package is installed
# install package if not installed
installPackage() {
retval=0
echo "Installing package $1"
if [ $(dpkg-query -W -f='${Status}' $1 2>/dev/null | grep -c "ok installed") -eq 0 ];
then
echo "Package $1 is not installed"
if echo "yes" | sudo apt-get install $1; then
retval=1
fi
else
echo "Package $1 is already installed"
retval=1
fi
return "$retval"
}

Manually Set Date Time

sudo date -s '2019-05-10 09:07:00'

Setup SSH Keys

mkdir -p /home/pi/.ssh
sudo echo '' | ssh-keygen -t rsa -q -N '' -f /home/pi/.ssh/id_rsa

Turn Off Strict Host Checking For a Domain

echo -e "Host google.comntStrictHostKeyChecking non" >> ~/.ssh/config

How to Automate Google Play Store Releases


Manual publishing of an app is time-consuming and involves a lot of repetitive steps that could be easily automated. Manual publishing involves these steps:

  • Create a signed APK for your app.
  • Log in to the Google Play developer account, select the app and create a release.
  • Upload the APK with a change-log.
  • Submit the update.

In this tutorial, I will outline how you can automatically upload APKs to the Google Play Store using Google Play Developer APIs. We will also see how you can use Travis CI to automatically increment the version, build the app, ship to the Play Store, and more.

Prerequisites

Before we get started with the tutorial, make sure that you have the following things already set up.

  • Google Play Developer Account.
  • An app listed in your developer account.
  • A CI tool for triggering automatic builds.

Note: I will use Travis CI for this tutorial as it is free to use with open source projects.

Signing your Android APK

To make a release to the Play Store, you need to sign the APK using a Keystore. Be sure to have the Keystore details specified in your app’s build.gradle file. You can use Android Studio’s inbuilt option (Build > Generate Signed APK…) to generate a Keystore or manually specify the Keystore details in the Gradle file.


Once you have created a Keystore, specify its details in the app’s build.gradle file. Add this snippet to your build.grade’s android section:

android {
...
signingConfigs {
release {
storeFile file("someDirectory/my_keystore.jks")
storePassword "my_store_pass_here"
keyAlias "my_key_alias_here"
keyPassword "my_key_pass_here"
}
}
buildTypes {
release {
signingConfig signingConfigs.release
minifyEnabled false
proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
}
}
}

After setting up the signing keys in the Gradle file, you can execute the following command to generate a signed version of the APK.

./gradlew :app:assembleRelease

For reference, you can check out the commit for configuring Signing keys in the sample app. These configs would later be used to automate the signing process of the app. Also for security purposes, we will encrypt these configs to prevent its malicious use.

Setup API Access for Automatic Publishing

The Google Play Developer Console provides API support for you to be able to push updates automatically. This ability allows you to trigger builds on your continuous integration server and have them uploaded the Play store for alpha or beta testing, as well as pushing to production directly.

Link Google Play Android Developer project Login into your Google Play Developer account and go into Settings > Developer account > API access. Here, link Google Play Android Developer under Linked Project section.


Create an OAuth Client and Service Account Next, you need to create an OAuth Client and a Service Account. Simply click on CREATE OAUTH CLIENT button, and it will create an OAuth Client. Clicking on CREATE SERVICE ACCOUNT will show a dialog indicating that you need to visit Google API Console and manually create a Service Account.


Click on the Create Service Account button on the Google API Console page and set the service account name.


Click on Create Key button to create a new private key and select P12 as key type. Download the P12 file while contains the service account details.


This .p12 file will be required for authenticating with Google Play Access API later on. Recheck the API Access page to make sure that the created service account appears on that page.


Click on Grant Access and make sure the checkboxes for Edit store listing, pricing & distribution, Manage Production APKs and Manage Alpha & Beta APKs are checked.


Include the Gradle Plugin in the Project

Now that API access has been enabled for the app, we can use it to publish our app using these APIs. Fortunately, there is already an open source Gradle Play Publisher plugin, that can be used to upload your App Bundle or APK and other app details to the Google Play Store.

Add the following in the top level Gradle file build.gradle file:

buildscript {
repositories {
jcenter()
}

dependencies {
// ...
classpath 'com.github.triplet.gradle:play-publisher:2.0.0-rc1'
}
}

Add the following to the top of your app/build.gradle file:

apply plugin: 'com.android.application'
apply plugin: 'com.github.triplet.play'
android {
signingConfigs {
release {
storeFile file("someDirectory/my_keystore.jks")
storePassword "my_store_pass_here"
keyAlias "my_key_alias_here"
keyPassword "my_key_pass_here"
}

}
buildTypes {
release {
signingConfig signingConfigs.release
minifyEnabled false
proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
}
}
}
play {
track = "alpha"
userFraction = 1
serviceAccountEmail = "SERVICE_ACCOUNT_NAME"
serviceAccountCredentials = file("../play.p12")
resolutionStrategy = "auto"
outputProcessor { // this: ApkVariantOutput
versionNameOverride = "$versionNameOverride.$versionCode"
}
}

Now, that you have setup Gradle Play Publisher plugin to your project, you can check what new Gradle tasks have been added. Execute ./gradlew tasks to review the tasks added by the plugin.


We can finally publish the app using this plugin by simply executing this task:

./gradlew publishReleaseApk

Check out the commit in the sample app to see changes done for Gradle Publisher Plugin integration. Note that if you try executing this task for an unreleased app, it will throw an error as shown in the screenshot.


This concludes the steps required for eliminating the manual release process. You still need to run this task for the actual release to happen. In the next, section we will go through the steps required for CI integration. The steps mentioned below are specific to Travis CI, but the process would be similar for other popular CIs like Jenkins or CircleCI.

CI Integration

To have Travis build a signed, release version of the APK, it needs the Keystore details. The challenge is to be able to do this securely- without adding the Keystore file or any other information in the source control or having them visible to anyone publicly.

Achieving automated releases requires the following:

  • Add an encrypted version of the Keystore file to the repository
  • Store Keystore’s credentials securely
  • Build and deploy a signed, release version of the APK when new code is pushed to the repository

Prerequisites

Setup the initial Travis Config Before, we dive into configuring Travis for automated releases, make sure you have .travis.yml configured for your project. My config looks like the snippet shown below:

language: android
addons:
apt:
packages:
- w3m
env:
global:
- ANDROID_TARGET=android-22
- ANDROID_ABI=armeabi-v7a
- ADB_INSTALL_TIMEOUT=12
jdk:
- oraclejdk8
android:
components:
- tools
- platform-tools
- build-tools-28.0.3
- extra-google-m2repository
- extra-android-m2repository
- android-22
- android-28
licenses:
- android-sdk-license-.+
script:
- "./gradlew clean build"

Check out the initial Travis config in the sample app.

Install and Log ****in to Travis CI Command Line Tools Firstly, install Travis Command line tools by executing:

gem install travis

Next, login to Travis CI using:

travis login --com

If you are still using travis-ci.org, you need to use --org.

Store Keystore and Service Account Key Files Securely with Travis

We already have our Keystore and Service Account key setup, and now we need to add them to Travis. We will be using its File Encryption functionality for it. Initially, we put the keystore.jks and service-account-key.p12 files in the root folder of the app. Encrypting a single file is quite simple and can be achieved by executing a single command.

travis encrypt-file keystore.jks --add

In case of multiple files, Travis CLI has an issue which overrides the existing secure environment variables when the command is invoked for a different file. So, as a workaround, we will first create an archive of the sensitive files and then store it on Travis. Run the following commands for it:

tar cvf secrets.tar keystore.jks service-account-key.p12

You’ll see something like the following output.


Before proceeding to the next step, delete the original Keystore, Service Account Key and secrets.tar files from the repository folder. Now you’re left with secrets.tar.enc – which is the encrypted version of the secrets.tar file. It can be safely committed to your repository. You will notice a before_install step added to your .travis.yml file.

before_install:
- openssl aes-256-cbc -K $encrypted_054692ad430f_key -iv $encrypted_054692ad430f_iv -in secrets.tar.enc -out secrets.tar -d

Also, your Travis CI Settings would start showing two encrypted environment variables corresponding to the Keystore file.


Notice that even we can’t see those values. They’re secret and only available to the Travis Virtual Machine when it works on your app. The command in the before_install step runs and decrypts the secrets.tar.enc file to generate the secrets.tar. Add another step to unarchive the tar file.

- tar xvf secrets.tar

Store the Credentials Securely with Travis

Travis also requires the credentials to create a signed APK. Go to the Travis web console settings page and add your parameters. Ensure that the Display Value in build log option is toggled off else they would be printed to build logs.


When you build the app in the Travis environment, key_alias, key_password, and keystore_password would be available as system variables.

Update Gradle File to Use Environment Variables

Next, you need to update the app’s build.gradle file to use these environment variables while building on CI. Note, that the secure environment variables(decrypted Keystore and p12 files) are not available for Pull Requests. We will add a check in our Gradle file to determine if the environment variables can be used or not.

apply plugin: 'com.android.application'
def isRunningOnTravisAndIsNotPRBuild = System.getenv("CI") == "true" && file('../play.p12').exists()
if(isRunningOnTravisAndIsNotPRBuild) {
apply plugin: 'com.github.triplet.play'
}
android {
.
.
signingConfigs {
release
}
.
.
if (isRunningOnTravisAndIsNotPRBuild) {
signingConfigs.release.storeFile = file("../keystore.jks")
signingConfigs.release.storePassword = System.getenv("keystore_password")
signingConfigs.release.keyAlias = System.getenv("key_alias")
signingConfigs.release.keyPassword = System.getenv("key_password")
}
}
.
.
if (isRunningOnTravisAndIsNotPRBuild) {
play {
track = "alpha"
userFraction = 1
serviceAccountEmail = "test@api-522342-837358.iam.gserviceaccount.com"
serviceAccountCredentials = file("../service-account-key.p12")
resolutionStrategy = "auto"
outputProcessor { // this: ApkVariantOutput
versionNameOverride = "$versionNameOverride.$versionCode"
}
}
}

System.getenv is one of the default variables Travis sets for us, and it lets us determine if the build is running on Travis CI or locally. If the build is running on Travis,

  • Apply the Gradle Play Publisher Plugin
  • Populate our signing config with the store file, password, key alias, and alias password.
  • Add the play config for Play Store releases

Update Travis Config to Release on Each Commit

Finally, update the Travis .yml file to run the publishReleaseApk task whenever a new commit is pushed to master. Add the following step in the script section of the .travis.yml file.

script:
- if [ "$TRAVIS_PULL_REQUEST" == "false" ] && [ "$TRAVIS_BRANCH" == "master" ]; then
./gradlew publishReleaseApk;
fi
  • TRAVIS_PULL_REQUEST determines if a Pull Request triggers the build. Remember, secure variables are not available for Pull Requests.
  • TRAVIS_BRANCH determines whether the branch is master. We do not want to make a release whenever a commit gets pushed to another branch.

Check out the commit in the sample app to see changes done for encrypting and storing the keys on Travis. You are all set now! Commit your changes and push to master so that a build gets triggered on Travis. Once the build completes successfully, an APK gets uploaded to alpha track on Google Play Store.

Conclusion

We have successfully eliminated all the manual steps required to make a Play Store release. Now, Travis can automatically publish a signed APK on every commit to master. We were able to quickly consume the Google Play Developer APIs owing to the open source Gradle Play Publisher plugin. Also, we went through the steps required to store Keystore and Service Account keys with Travis securely.

We recently automated our alpha releases for Wikimedia Commons Android app and it has immensely helped us in making around automatic 100 releases in December 2018 and collecting feedback from our Alpha users. It would have been quite difficult to make 100 manual releases and would have hampered the speed of closed group testing. For any fast-paced development environment, it is quite essential to eliminate the manual steps to boost developer productivity.

You can find all the code used in this article in this example project on Github.

Querying Wikimedia Commons DB to get User Statistics

Getting User Statistics on Wikimedia Commons

Wikimedia Commons is an online repository of free-use images, sounds, and other media files. It is a project of the Wikimedia Foundation.

If you are a Commons user or need to get statistics for a particular user, then you can query the commonswiki DB to get the results. In this post, I will introduce you to a few SQL queries to get various user statistics.

Get Count of File Uploads

You can check the number of file uploads by a user using the following query:

use commonswiki_p;
select
count(*)
from
logging_userindex
where
log_type = "upload"
and log_user =(
select
user_id
from
user
where
user_name = "Maskaravivek"
);

You can run the query on Quarry.

https://quarry.wmflabs.org/query/31757

Get Count of Deleted file uploads

Commons has a strict deletion policy and the community can nominate a picture for deletion if your upload violates any of the policies.

You can use the following query to get the count of deleted file uploads.

select count(*) from commonswiki_p.filearchive_userindex where fa_user_text="Maskaravivek"

commonswiki_p.filearchive_userindex has an index on username and querying it is much faster than querying the original table ie. commonswiki_p.filearchive .

You can run the query on Quarry.

https://quarry.wmflabs.org/query/31757

Get Count of Articles Using Images

The whole purpose of the Commons image repository is to let Wikipedia articles use the images.

The query below gives the count of articles using images from a particular user.

select
count(*) as articlesUsing
from
commonswiki_p.globalimagelinks
where
gil_to in (
select
log_title
from
commonswiki_p.logging_userindex
where
log_type = "upload"
and log_user =(
select
user_id
from
commonswiki_p.user
where
user_name = "Maskaravivek"
)
);

You can run the query on Quarry.

https://quarry.wmflabs.org/query/31757

Get Count of Unique Images USed

The query below gives the count of articles using images from a particular user.

select
count(distinct gil_to) as uniqueUsed
from
commonswiki_p.globalimagelinks
where
gil_to in (
select
log_title
from
commonswiki_p.logging_userindex
where
log_type = "upload"
and log_user =(
select
user_id
from
commonswiki_p.user
where
user_name = "Maskaravivek"
)
);

You can run the query on Quarry.

https://quarry.wmflabs.org/query/31757

Get Count of Images Edited by Someone else

The pictures that you upload to commons can be edited by some other user as well. The following query can be used to get the count of such images.

use commonswiki_p;
select
count(*)
from
revision
where
rev_page in (
select
log_page
from
logging_userindex
where
log_type = "upload"
and log_user =(
select
user_id
from
user
where
user_name = "Maskaravivek"
)
)
and rev_user !=(
select
user_id
from
user
where
user_name = "Maskaravivek"
)
group by
rev_page
having
count(*) > 1

You can try the query on Quarry.

https://quarry.wmflabs.org/query/31757

Get Number of Thanks received

Users can express thanks to other users using Commons.

The following query can be used to get the count of the number of thanks received by a user.

use commonswiki_p;
select count(*) from logging_logindex where log_type="thanks" and log_title="Maskaravivek";

https://quarry.wmflabs.org/query/31757

The Commons Android app shows these statistics in a very nice interface.

User Achievements

Make sure you give this post 50 claps and follow me if you enjoyed this post and want to see more!

How to get started with Open Source?

Open Source by Nick Youngson CC BY-SA 3.0 Alpha Stock Images

There are a lot of developer and wiki guides to set up and get started with your first open source contribution. As a beginner, I always found these wikis a bit overwhelming as they suddenly introduce you to a lot of options and projects. As a beginner, I wasn’t looking for lots of options but rather wanted someone to hand hold me and help me find one project which was easy to start with open source.

Throughout my college days, open source fascinated me and I always wanted to contribute to some open source project but couldn’t find the right project. When I was in the second year of college I made a failed attempt to kick start my open source journey by applying for Google Summer of Code. I attribute my failure to these reasons:

  • I simply wanted to land an internship with GSoC and could never zero down on a suitable project to get started.
  • I found the developer guides overwhelming as they gave me a lot of options and I wasn’t able to decide what was best for me.
  • I didn’t dedicate enough time to methodically get started with open source. Rather than focus on finding the right project, I focussed on making the first patch and submitting a proposal.

It’s also very important to ask yourself why you want to contribute to open source before actually making a contribution. The answer might be different for different individuals but it should be convincing enough for you to make the choice.

In this post, I will share some tips on how to choose a project and make your first contribution to open source.

Why contribute to open source?

  • Improve your existing coding/design skills
  • Meet people who are interested in similar things. Open source communities are more often than not very welcoming and you might soon form lifelong friendships through your participation be it in online chats or running into each other at conferences.
  • Personally, I believe that having a mentor and mentoring someone is very important to learn continuously. Open source can fulfill both these needs.
  • It’s empowering to make even a very tiny contribution. I have made 100s of contributions to open source but am most proud of my little contribution on Wikipedia where I added an image to Parle-G’s article from Wikimedia Common’s image repository.

How to choose a project?

There are lots of open source projects out there, but for a beginner, your options might be limited. Most of the popular projects on Github are large codebases which might be a bit overwhelming for a beginner.

Personally, my criteria for choosing a project is based on the following factors:

Active Project

At first glance, the pulse as shown on Github can be a good indicator to check whether the project is being actively maintained or not.

A look at the number of open/closes issues, and open/merged pull requests give a fair idea about whether the project is still actively maintained.

Beginner Friendly

It should be a beginner friendly project. Project’s wiki is a good indication of how welcome new-comers are to the project. Project maintainers have to make lots of efforts to make their projects appealing to new-comers. Github has a guide for project maintainers that helps them make their project beginner friendly.

Here’s how Wikimedia Commons app’s community profile looks . We made an effort to tick off most of the items in the checklist so that its easier to get started for newcomers.


Some projects are open source but are not beginner friendly and have just a core group of developers who make contributions. For eg., some projects already have a core team of full-time developers who are paid to maintain the project. Such a project will people assigned to most of the issues and newcomers might have limited opportunities to make impactful contributions to it.

Coding Language

It should be in a language that I am comfortable with, eg. Java, Python. Rather than getting familiar with an entirely new language to contribute to a project, it’s much simpler to choose a project which uses a language you already know. I make most of my contributions to Wikimedia Commons Android app as I am already familiar with Java and Android. It makes it much easier to understand the existing code base and make contributions to it.

Project Popularity

It should be reasonably popular. The number of stars/forks a project has can be a fair index to gauge the popularity. Naturally, a project like Linux with 780K commits and 62K stars would be overwhelming for a beginner. I would instead choose a project with a few thousand commits and 100–5000 stars. I believe the chances for it being beginner friendly is much more when compared with an immensely popular project.

Also, taking a look at other statistics like last commit, contributors, traffic, code frequency, etc. can give a fair idea about the project’s popularity and its activity.

Hosted on Github

Also, it should preferably be hosted on Github. If the project uses some other tool for code review, for instance, Gerrit then its a bit of turn off for me as it requires additional steps for setting up the environment. For eg., most of Wikimedia projects use Gerrit for code review. I never found it appealing enough to set it up and start using it. I agree that it’s not a significant factor while deciding on the project but I prefer Github for the entire workflow.


Conclusion

Once you have zeroed down on the project that you want to contribute to, the next step is finding the first issue that you can pick up and fix. Keep in mind that you don’t have to stick to the project forever if you don’t like it. The project is meant to make you get started with the journey. Open source guide has a few checklists that could help you to get started with your open source journey.

Here are some curated lists of beginner-friendly projects/issues:


Make sure you give this post 50 claps and follow me if you enjoyed this post and want to see more!

Introduction to Async-Await in Javascript

Introduction to Async-Await in Javascript

Today I learned that Javascript has a special syntax for working with promises in a more comfortable fashion. It’s the async and await which was originally introduced in C#. I am not an expert in Javascript but am quite excited to share the basics about async and await.

Async

async before a function means that the function always returns a promise. If the code has a non-promise return type then Javascript automatically wraps it into a resolved promise with that value.

Let’s see how a normal promise looks in javascript.

var promise = new Promise(function(resolve, reject) {
setTimeout(function() {
resolve('hello world');
}, 300);
});
promise.then(function(value) { 
console.log(value);
});
console.log(promise);

Here’s a simple example of a function with async keyword.

async function myFunc() {
return "Hello world"
}
myFunc().then(data => console.log(data))

Executing the above function gives the following result.

Executing the Async function

Read more about Async function here.

Await

await makes Javascript wait until that promise settles and returns its result. It can only be used inside an async function. The promise could either resolve or get rejected and accordingly the following happens:

  • The execution of the async function continues once the promise is resolved and the value of the await expression is that of the fulfilled promise.
  • If the promise is rejected, the await expression throws the rejected value.

Here’s an example of await in a function.

async function myFunc() {
let promise = new Promise((resolve, reject) => {
setTimeout(() => resolve("hello world!"), 1000)
});
let result = await promise;
console.log(result);
}
myFunc();

This is how the output looks:

Response for a function that uses await

Note: You cannot use await in a normal function. It can be used only in an async function.

Read more about Await keyword here.

How to Mitigate Slow Build Times While Using Firebase Performance Plugin

I recently added Firebase Performance Plugin to one of my projects and experienced a drastic increase in the build time of the app. The app initially used to build in under 20 seconds and with the addition of this plugin, it started taking around 5–6 minutes for a single build. This was quite annoying and I looked for what might be causing the increase in build times.

If you look closely you will find out that the following task is taking too long to build:

app:transformClassesWithFirebasePerformancePluginForDebug

There is a post-compilation phase when using Firebase Performance on Android that results in slow build times.

How to Mitigate the Issue

The fix that applied only mitigates the issue by adding a parameter to build command so that this plugin can be disabled during development.

In root/buildscript/dependencies wrap the plugin classpath inside the following if condition.

if (!project.hasProperty("disable-performance-plugin")) {
classpath('com.google.firebase:firebase-plugins:1.1.5') {
exclude group: 'com.google.guava', module: 'guava-jdk5'
}
}

Excluding com.google.guava depends on whether it is causing conflicts with your Guava dependencies as mentioned in the getting started documentation.

Next, in app/build.gradle wrap the apply plugin code in the following condition.

if (!project.hasProperty("disable-performance-plugin")) { 
apply plugin: 'com.google.firebase.firebase-perf'
}

Now, simply build using the command line with the parameter for disabling the plugin.

./gradlew your-task -Pdisable-performance-plugin

If you use Android Studio for building the project then you add the same config in Compiler settings of Studio. Set the command line options to:

-Pdisable-performance-plugin

Set the Command-line option

That’s it. Adding this parameter should make your life easier! 🙂

Credits to Ivan Kravchenko for the original answer on Stack Overflow.


Make sure you give this post 50 claps and follow me if you enjoyed this post and want to see more!

Understanding the format of NDEF Messages

The NFC Data Exchange Format (NDEF) is a standardized data format that can be used to exchange information between any compatible NFC device and another NFC device or tag. The data format consists of NDEF Messages and NDEF Records.

In this series of articles, I will explain how an NDEF message can be constructed and stored on an NFC tag. Assume a company wants to issue tags to that can be used in public transport systems as a replacement of paper tickets. These tags can be tapped on an NFC enabled Android device which will scan the tag and register entry and exit time of the user. I won’t be going into how the application logic should be constructed but will be dealing with how the required data can be put on the tags.

In this article lets get an overview of how NDEF messages are constructed.

NDEF Message

NDEF messages are the basic transport mechanism for NDEF records, with an NDEF message containing one or more NDEF records.

For this use case the tag primarily needs the following records on it:

  • A user identifier, say username to identify the user.
  • An application identifier, to tell the android system which app will be handling the NFC tag.

NDEF Record

NDEF Records contain a specific payload and have the following structure that identifies the contents and size of the record. The basic elements that construct an NDEF record are:

  • TNF ie. Type Name Format Field
  • type which is a value corresponding to bits set in TNF field
  • ID value
  • payload value

More specifically an NDEF record can be composed in the following way.

Bit 7     6       5       4       3       2       1       0
------ ------ ------ ------ ------ ------ ------ ------
[ MB ] [ ME ] [ CF ] [ SR ] [ IL ] [ TNF ]
[                         TYPE LENGTH                        ]
[                       PAYLOAD LENGTH                       ]
[                          ID LENGTH                         ]
[                         RECORD TYPE                        ]
[                              ID                            ]
[                           PAYLOAD                          ]

NDEF record explained

An NDEF record starts with 8 bits header that describes the record.

TNF: The Type Name Format or TNF Field of an NDEF record is a 3-bit value that describes the record type.

https://gist.github.com/maskaravivek/2b33a6df989ab31c0169796dc7483675

IL [ID Length bit]: The IL flag indicates if the ID Length Field is present or not.

SR [Short Record bit]: The SR flag is set to 1 if the payload length field is 1 byte (8 bits/0–255) or less. This allows for more compact records.

CF [Chunk Flag bit]: The CF flag indicates if this is the first record chunk or a middle record chunk. For the 1st record of the message, it is set to 0 and for subsequent records, it is set to 1.

ME [Message End bit]: The ME flag indicates if this is the last record in the message. It is set to 1 for the last record.

MB [Message Begin bit]: The MB flag indicates if this is the first record in the message. It is set to 1 for the first message.

NDEF Records

For our application, we need the following NDEF records.

Tag data

  • tnf: TNF_MIME_MEDIA,
  • type: application/vnd.com.tickets [in bytes]
  • id: null
  • payload: byjwdH_6DNsuU4iTSrVaNEe6e52VLhhr4v_iGTRP7jQ= [in bytes]

Android Application Record(AAR)

  • tnf: TNF_EXTERNAL_TYPE
  • type: android.com:pkg [in bytes]
  • id: null
  • payload: com.example.tickets [in bytes]

While working on Android, the following constructor can be used to construct an NdefRecord.

public NdefRecord (short tnf, 
byte[] type,
byte[] id,
byte[] payload)

With multiple NdefRecord you can construct an NdefMessage.

public NdefMessage (NdefRecord record, 
NdefRecord... records)

We have seen an overview of the different parts of an NDEF message. In the next part of this article, we will see how these two NDEF records can be written on a tag using Android’s NFC capabilities.

We will also discuss the memory structure of two kinds of tags ie. Mifare Ultralight C and Mifare Classic.


Make sure you give this post 50 claps and follow me if you enjoyed this post and want to see more!

How To Improve Coverage for your Android App Using Mockito and Espresso — Part 2


In the first part of this article we got an introduction about various frameworks available to us for writing tests for an Android app. We also saw some best practices that could be followed to write more testable code. Now lets write some actual code and then add unit and instrumentation tests for the same.

In the next few, sections we would be using examples from a really simple application that I built for the purpose of this tutorial. The app has an EditText that takes a username as input and displays the name in a TextView on click of a button. Feel free to take the complete source code for the project from Github. Here’s a screenshot of the app.


Writing Local Unit Tests

Unit tests can be run locally on the development machine without a device or an emulator. This testing approach is efficient because it helps you avoid the overhead of loading the target app and unit test code onto a physical device or emulator every time your test is run. In addition to Mockito, you will also need to configure the testing dependencies for your project to use the standard APIs provided by the JUnit 4 framework.

Setting up the Development Environment

Start by adding a dependency on JUnit4 in your project. The dependency is of type testImplementation which means that the dependencies are only required to compile the test source of the project.

testImplementation 'junit:junit:4.12'

We will also need Mockito library to make interacting with Android dependencies easier.

testImplementation "org.mockito:mockito-core:$MOCKITO_VERSION"

Make sure to sync the project after adding the dependency. Android studio should have created the folders structure for unit tests by default, if not make sure the following directory structure exists.

<Project Dir>/app/src/test/java/com/maskaravivek/testingExamples

Creating your First Unit Test

Suppose you want to test the displayUserName function in the UserService. For the sake of simplicity, the function simply formats the input and returns it back. In a real-world application, it could make a network call to fetch the user profile and return the user’s name.

https://gist.github.com/maskaravivek/6d64495bbec8f361cbfdc2b8c707bea2

We will start by creating a UserServiceTest class in our test directory. The UserService class uses Context which needs to be mocked for the purpose of testing. Mockito provides a @Mock notation for mocking objects which can be used as follows,

@Mock internal var context: Context? = null

Similarly, you need to mock all dependencies required to construct the instance of the UserService class. Before your test, you need to initialize these mocks and inject it into the UserService class.

  • @InjectMock creates an instance of the class and injects the mocks that are marked with the annotations @Mock into it.
  • MockitoAnnotations.initMocks(this); initializes fields annotated with Mockito annotations.

Here’s how it can be done.

https://gist.github.com/maskaravivek/32174f5b847ebb1ff020c17c69818488

Now you are done setting up your test class. Let’s add a test to this class that verifies the functionality of displayUserName function. Here’s how the test looks,

https://gist.github.com/maskaravivek/930b97d3f5e398d6df33bc50f604285c

The test uses a doReturn().when() statement to provide a response when a context.getString() is invoked. For any input integer, it will return the same result "Hello %s!". We could have been more specific by making it return this response only for a particular string resource ID but for the sake of simplicity we are returning the same response to any input. Finally, here’s how the test class looks,

https://gist.github.com/maskaravivek/a2a72c014c6340ddbba7093f13aa4a77

Running your Unit Tests

In order to run the unit tests, you need to make sure that Gradle is synchronized. In order to run a test click on the green play icon in the IDE.


When the unit tests are run, successfully or otherwise, you should be able to see this in the run menu at the bottom of the screen.


You are done with your first unit test!

Writing Instrumentation Tests

Instrumentation tests are most suited for checking values of UI components when an activity is run, for instance in the above example we want to make sure that the TextView shows the correct username after the Button is clicked. They run on physical devices and emulators and can take advantage of the Android framework APIs and supporting APIs, such as the Android Testing Support Library. We’ll use espresso to take actions on the main thread such as button clicks, text change etc.

Setting up the Development Environment

Add a dependency on espresso:

androidTestImplementation 'com.android.support.test.espresso:espresso-core:3.0.1'

Instrumentation tests are created in an androidTest folder.

<Project Dir>/app/src/androidTest/java/com/maskaravivek/testingExamples

If you want to test a simple activity, you should create your test class in the same package as your Activity.

Creating your First Instrumentation Test

Let’s start by creating a simple activity that takes a name as input and on a click of a button it sets displays the username. The activity looks like: The code for the above activity is quite simple.

https://gist.github.com/maskaravivek/4a6bbaea33beb8e21d852aa1790d11f7

To create a test for the MainActivity we will start by creating a MainActivityTest class under the androidTest directory. Add the AndroidJUnit4 annotation to the class to indicate that the tests in this class will use the default Android test runner class.

@RunWith(AndroidJUnit4::class) class MainActivityTest {}

Next, add an ActivityTestRule to the class. This rule provides functional testing of a single Activity. During the duration of the test, you will be able to manipulate your Activity directly using the reference obtained from getActivity().

@Rule @JvmField var activityActivityTestRule = ActivityTestRule(MainActivity::class.java)

Now that you are done setting up the test class, let’s add a test that verifies that the username is displayed by clicking the Set User Name button.

https://gist.github.com/maskaravivek/2d2abd5b893a5088c90f6adbec43f3f4

The above test is quite simple to follow. It first simulates typing some text in the EditText, performs the click action on the button and then checks whether the correct text is displayed in the TextView.

The final test class looks like,

https://gist.github.com/maskaravivek/4f4d854d29ad10e4eb88e690a236549b

Running your Instrumentation Tests

Just like for unit tests, click on the green play button in the IDE to run the test.


On clicking the play button, the test version of the app is installed on the emulator or device and the test runs automatically on it.


Intrumentation Testing Using Dagger, Mockito and Espresso

Espresso is one of the most popular UI testing frameworks with good documentation and community support. Mockito ensures that objects perform the actions that are expected from them. Mockito also works well with dependency injection libraries like Dagger. Mocking the dependencies will allow us to test a scenario in isolation. Until now our MainActivity isn’t using any dependency injection and as a result of it, we were able to write our UI test very easily. To make things a bit interesting let’s inject UserService in the MainActivity and use it to get the text to be displayed.

https://gist.github.com/maskaravivek/b528a79dc14fca67703d2f24b492ac08

With Dagger in the picture, we will have to set up a few things before we write instrumentation tests. Imagine the displayUserName function internally uses some API to fetch the details of the user. There should not be a situation in which tests do not pass due to server fault. To avoid such situation we can use dependency injection framework Dagger and for networking Retrofit.

Setting up Dagger in the Application

We will quickly set up the basic modules and components required for Dagger. If you are not familiar with Dagger, check out Google’s documentation on it. We will start adding dependencies for using Dagger in the build.gradle file.

https://gist.github.com/maskaravivek/dd19a5167ae8eb798e185c1a940610dc

Create a component in the Application class, add necessary modules which will be used in our project. We need to inject dependencies in the MainActivity of our app. We will add a @Module for injecting in the activity.

https://gist.github.com/maskaravivek/4e04dbda6f3539a1a53f67dabdae5219

The AppModule class will provide the various dependencies required by the application. For our example, it will just provide an instance of Context and UserService.

https://gist.github.com/maskaravivek/c12b491987012a9e6812db1e8c2cf7fa

The AppComponent class lets you build the object graph for the application.

https://gist.github.com/maskaravivek/b6a566c187543c1c8f62a8f796cbf7c5

Create a method that returns already built component, then inject this component into onCreate().

https://gist.github.com/maskaravivek/c2d03d1ac1fc4d01902f6ee709c25bc7

Setting up Dagger in the Test Application

In order to mock responses from the server, we need to create a new Application class that extends the above class.

https://gist.github.com/maskaravivek/fc9abb66e6950470e9cef718635671b0

As you can see in the example above we used Mockito to mock UserService and assume results. We still need a new runner that will point to new application class with overwritten data.

https://gist.github.com/maskaravivek/026efea86022fca28da94677787dad40

Next, you need to update the build.gradle file to use the MockTestRunner.

https://gist.github.com/maskaravivek/40cbab711b98608b5b76fea2f4b079dc

Running the Test

All tests with new TestExamplesApplication and MockTestRunner should be added at androidTest package. This implementation makes the tests fully independent from the server and gives the ability to manipulate responses. With the above setup in place, our test class won’t change at all. When the test is run, the app uses TestExamplesApplication instead of ExamplesApplication and thus a mocked instance of UserService gets used.

https://gist.github.com/maskaravivek/03df0c9bb9d6156e4e98fa70ad7561f1

The test runs successfully when you click on the green play button in the IDE.


That’s it, you have successfully setup Dagger and ran tests using Espresso and Mockito.

Conclusion

The article highlights that the most important aspect of improving code coverage is to write testable code. Frameworks like Espresso and Mockito provide easy to use APIs which makes writing tests for various scenarios easier. Tests should be run in isolation so mocking the dependencies gives us an opportunity to ensure that objects perform the actions that are expected from them. There are a variety of Android testing tools available, and as the ecosystem matures, the process of setting up a testable environment and writing tests will become easier. Writing testable code requires some discipline, concentration, and extra effort. As a reward, we’ll end up with clean, easy-to-maintain, loosely coupled, and reusable APIs, that won’t damage developers’ brains when they try to understand it. The complete source code for the examples used in this article is available on Github. Feel free to take a look at it.


Make sure you give this post 50 claps and follow me if you enjoyed this post and want to see more!

How To Improve Coverage for your Android App Using Mockito and Espresso — Part 1


In app development, a variety of use cases and interactions come up as one iterates the code. The app might need to fetch data from a server, interact with the device’s sensors, access local storage, or render complex user interfaces. The important thing to consider while writing tests is the units of responsibility that emerge as you design the new feature. The unit test should cover all possible interactions with the unit including standard interactions and exceptional scenarios. In this article, we will cover the fundamentals of testing and frameworks like Mockito and Espresso that developers can use to write unit tests. In the second part of the article we will gets hand on and write a sample application with unit and instrumentation tests.

Fundamentals Of Testing

A typical unit test contains 3 phases.

  • First, the unit test initializes a small piece of an application it wants to test.
  • Then it applies some stimulus to the system under test usually by calling a method on it
  • Finally, it observes the resulting behavior.

If the observed behavior is consistent with the expectations, the unit test passes, otherwise, it fails, indicating that there is a problem somewhere in the system under test. These three unit test phases are also known as Arrange, Act, and Assert, or simply AAA. The app should ideally include three categories of tests: small medium and large.

  • Small tests comprise of unit tests that mock every major component and run quickly in isolation
  • Medium tests are integration tests that integrate several components and run on emulators or real devices
  • Large tests are integration and UI tests that run by completing a UI workflow and ensure that the key end-user tasks work as expected.

Writing small tests allows you to address failures quickly but it’s difficult to gain confidence that a passing test allows your app to work. It’s important to have tests from all categories in the app, though the proportion of each category can vary from app to app. A good unit test should be easy to write, readable, reliable and fast.
 Here’s a brief introduction to Mockito and Espresso that make testing Android apps easier.

Mockito

There are various mocking frameworks but the most popular of them all is Mockito!

Mockito is a mocking framework that tastes really good. It lets you write beautiful tests with a clean & simple API. Mockito doesn’t give you hangover because the tests are very readable and they produce clean verification errors.

Its fluent API separates pre-test preparation from post-test validation. Should the test fail, Mockito makes it clear to see where our expectations differ from reality! The library has everything you need to write complete tests.

Espresso

Espresso helps you write concise, beautiful, and reliable Android UI tests. The code snippet below shows an example of an Espresso test. We will take up the same example again later in this tutorial when we talk in detail about instrumentation tests.

https://gist.github.com/maskaravivek/4c74c6a55b1197188a45a26f103cb05e

Espresso tests state expectations, interactions, and assertions clearly without the distraction of boilerplate content, custom infrastructure, or messy implementation details getting in the way. Whenever your test invokes onView(), Espresso waits to perform the corresponding UI action or assertion until the synchronization conditions are met ie.

  • The message queue is empty
  • No instances of AsyncTask is currently executing a task
  • The idling resources are idle

These checks ensure that the test results are reliable and dependable.

Writing Testable Code

Unit testing Android apps is difficult and sometimes impossible. Only and only a good design can help you making unit testing easier. Here are some of the concepts that are important for writing testable code.

Avoid Mixing Object Graph Construction with Application Logic.

In a test, you want to instantiate the class under test and apply some stimulus to the class and assert that the expected behavior was observed. Make sure that the class under test doesn’t instantiate other objects and those objects do not instantiate more objects and so on. In order to have a testable code-base, your application should have two kinds of classes.

  • The factories, these are full of the “new” operators and are responsible for building the object graph of your application.
  • The application logic classes which are devoid of the “new” operator and are responsible for doing work.

Constructors Should Not Do Any Work

The most common operation you will do in tests is the instantiation of object graphs, so make it easy on yourself and make the constructors do no work other than assigning all of the dependencies into the fields. Doing work in the constructor will not just affect the direct tests of the class, but will also affect related tests which try to instantiate your class indirectly.

Avoid Static Methods Wherever Possible

The key to testing is the presence of places where you can divert the normal execution flow. Seams are needed so that you can isolate the unit of test. If you build an application with nothing but static methods you have procedural application. How much a static method will hurt from a testing point of view depends on where it is in your application call graph. A leaf method such as Math.abs() is not a problem since the execution call graph ends there. But if you pick a method in a core of your application logic than everything behind the method becomes hard to test, since there is no way to insert test doubles

Avoid Mixing of Concerns

A class should be responsible for dealing with just one entity. Inside a class a method should be responsible for doing just one thing. For eg. BusinessService should be responsible just for talking to a Business and not BusinessReceipts. Moreover, a method in BusinessService could be getBusinessProfile, but a method such as createAndGetBusinessProfile would not be ideal for testing. Solid design principles must be followed for good design like:

  • S — Single-responsibility principle
  • O — The open-closed principle
  • L — Liskov substitution principle
  • I — Interface segregation principle
  • D — Dependency Inversion Principle

We now have a fair understanding of why unit testing is important in an Android app and what are the best practices that could be followed to write testable code. In the second part of this tutorial we will write a sample application to put the above philosophies into action.


Make sure you give this post 50 claps and follow me if you enjoyed this post and want to see more!

Fetching Featured Images from Wikimedia Commons

Recently featured image on Commons: https://bit.ly/2IyitSG

Wikimedia Commons features many images regularly on its main page. As described by Commons,

Featured pictures are images from highly skilled photographers and illustrators that the Wikimedia Commons community has chosen as some of the highest quality on the site.

Getting an image featured is quite difficult as the statistics suggest.

There are currently 11,192 of such images in the Commons repository which is roughly 0.024% of the available images (46,591,843).

In the Wikimedia Commons Android app, we thought showing these featured images in the app would be a good addition as it would:

  • Show users what a great Commons picture is
  • Inspire/motivate users to take part in the same project
  • Let users hope that one day their picture will be shown at that place (in fact, show them that taking great pictures is worth the effort)

I started exploring the API that would let us fetch featured images and then we could consume it in the app. I came across the MediaWiki Generator API.

Check out the following API call that returns the first 10 features images from the API. https://bit.ly/2KRXg4l

Here’s a screenshot from the app showing the featured images from Commons:

Featured Images Screen

Let us discuss the parameters that the API takes:

  • action: query which is common param for all MediaWiki API calls
  • generator: we are going to generate a list of categorymembers
  • gcmtype: this param’s value is file as we want to extract information of the Commons image files
  • gcmtitle: this is name of the category under which all featured images on Commons are stored. Category:Featured_pictures_on_Wikimedia_Commons
  • prop: Setting the value of prop to imageinfo adds the imageinfo block in the API response. It contains the url and extmetadata fields that are required for getting some of the properties.

We wanted to display all the images in a category instead of showing just the first 10. So we used the continue query of MediaWiki API. To get more data, you use the query-continue value in the response.

The query-continue node will contain a subnode for each module used in the query that needs continuation, and these subnodes will contain properties to be used when making the followup “continuation” query. Note that clients should not be depending on the particular property names given for continuation of any module or the format of the values returned for continuation, as these may change.

Continue query: https://bit.ly/2Idkhgt

Moreover, the implementation in generic enough to load images from any category of Commons.

CategoryImagesActivity.startYourself(this, "Page title", "Category name");

Check out the full implementation for this feature here:

https://github.com/commons-app/apps-android-commons/pull/1456

The changes are now merged to master and you could take a pull of the code base and run it for yourself. 🙂

https://github.com/commons-app/apps-android-commons/pull/1456


Make sure you give this post 50 claps and follow me if you enjoyed this post and want to see more!

Bitnami