NPE when reading Calendar on Android

[simplysoft]

Currently we are experiencing problems when deserializing Calendar instance on Android. The stack trace looks like this:

Caused by: java.lang.NullPointerException: Attempt to read from null array
at org.nustaq.serialization.FSTObjectInput.readObjectWithHeader(FSTObjectInput.java:353)
at org.nustaq.serialization.FSTObjectInput.readObjectInternal(FSTObjectInput.java:327)
at org.nustaq.serialization.serializers.FSTMapSerializer.instantiate(FSTMapSerializer.java:77)
at org.nustaq.serialization.FSTObjectInput.instantiateAndReadWithSer(FSTObjectInput.java:497)
at org.nustaq.serialization.FSTObjectInput.readObjectWithHeader(FSTObjectInput.java:366)
at org.nustaq.serialization.FSTObjectInput.readObjectInternal(FSTObjectInput.java:327)
at org.nustaq.serialization.FSTObjectInput$2.readFields(FSTObjectInput.java:1096)
at org.nustaq.serialization.FSTObjectInput$MyObjectStream.readFields(FSTObjectInput.java:1359)
at java.util.Calendar.readObject(Calendar.java:1398)

The crash happens when trying to deserialize a key of the serialized hash map (of the readFields). Based on our testing, it looks like SimpleTimeZone is triggering the crash, when we use another time zone instance (e.g. libcore.util.ZoneInfo), there is no such crash.

Here is a sample test which reproduces that problem (testFST crashes, testJDK works) :

public class FastSerializationTest extends AndroidTestCase {

  private static class Container implements Serializable{
    Calendar calendar;

    public Container(Calendar calendar) {
      this.calendar = calendar;
    }
  }

  @Test public void testFST() throws Exception {
    FSTConfiguration conf = FSTConfiguration.createAndroidDefaultConfiguration();
    File testFile = File.createTempFile("fast-serialization", "test");

    Calendar calendar = Calendar.getInstance();
    TimeZone timeZone = TimeZone.getTimeZone("GMT+2:00"); // SimpleTimeZone causes crash
    // timeZone = TimeZone.getTimeZone("Europe/Zurich"); Works
    calendar.setTimeZone(timeZone);
    Container container = new Container(calendar);

    FileOutputStream stream = new FileOutputStream(testFile);
    FSTObjectOutput out = conf.getObjectOutput(stream);
    out.writeObject(container, Container.class);
    out.flush();
    stream.close();

    FileInputStream inStream = new FileInputStream(testFile);
    FSTObjectInput in = conf.getObjectInput(inStream);
    Container result = (Container) in.readObject(Container.class);
    stream.close();

    assertEquals(container.calendar, result.calendar);
  }

  @Test public void testJDK() throws Exception {
    File testFile = File.createTempFile("serialization", "test");

    Calendar calendar = Calendar.getInstance();
    TimeZone timeZone = TimeZone.getTimeZone("GMT+2:00");
    calendar.setTimeZone(timeZone);
    Container container = new Container(calendar);

    FileOutputStream stream = new FileOutputStream(testFile);
    ObjectOutputStream out = new ObjectOutputStream(stream);
    out.writeObject(container);
    out.flush();
    stream.close();

    FileInputStream inStream = new FileInputStream(testFile);
    ObjectInputStream in = new ObjectInputStream(inStream);
    Container result = (Container) in.readObject();
    stream.close();

    assertEquals(container.calendar, result.calendar);
  }
}

The SimpleTimeZone read & writeObject look like this:

private void writeObject(ObjectOutputStream stream) throws IOException {
        int sEndDay = endDay, sEndDayOfWeek = endDayOfWeek + 1, sStartDay = startDay, sStartDayOfWeek = startDayOfWeek + 1;
        if (useDaylight
                && (startMode != DOW_IN_MONTH_MODE || endMode != DOW_IN_MONTH_MODE)) {
            Calendar cal = new GregorianCalendar(this);
            if (endMode != DOW_IN_MONTH_MODE) {
                cal.set(Calendar.MONTH, endMonth);
                cal.set(Calendar.DATE, endDay);
                sEndDay = cal.get(Calendar.DAY_OF_WEEK_IN_MONTH);
                if (endMode == DOM_MODE) {
                    sEndDayOfWeek = cal.getFirstDayOfWeek();
                }
            }
            if (startMode != DOW_IN_MONTH_MODE) {
                cal.set(Calendar.MONTH, startMonth);
                cal.set(Calendar.DATE, startDay);
                sStartDay = cal.get(Calendar.DAY_OF_WEEK_IN_MONTH);
                if (startMode == DOM_MODE) {
                    sStartDayOfWeek = cal.getFirstDayOfWeek();
                }
            }
        }
        ObjectOutputStream.PutField fields = stream.putFields();
        fields.put("dstSavings", dstSavings);
        fields.put("endDay", sEndDay);
        fields.put("endDayOfWeek", sEndDayOfWeek);
        fields.put("endMode", endMode);
        fields.put("endMonth", endMonth);
        fields.put("endTime", endTime);
        fields.put("monthLength", GregorianCalendar.DaysInMonth);
        fields.put("rawOffset", rawOffset);
        fields.put("serialVersionOnStream", 1);
        fields.put("startDay", sStartDay);
        fields.put("startDayOfWeek", sStartDayOfWeek);
        fields.put("startMode", startMode);
        fields.put("startMonth", startMonth);
        fields.put("startTime", startTime);
        fields.put("startYear", startYear);
        fields.put("useDaylight", useDaylight);
        stream.writeFields();
        stream.writeInt(4);
        byte[] values = new byte[4];
        values[0] = (byte) startDay;
        values[1] = (byte) (startMode == DOM_MODE ? 0 : startDayOfWeek + 1);
        values[2] = (byte) endDay;
        values[3] = (byte) (endMode == DOM_MODE ? 0 : endDayOfWeek + 1);
        stream.write(values);
    }

    private void readObject(ObjectInputStream stream) throws IOException,
            ClassNotFoundException {
        ObjectInputStream.GetField fields = stream.readFields();
        rawOffset = fields.get("rawOffset", 0);
        useDaylight = fields.get("useDaylight", false);
        if (useDaylight) {
            endMonth = fields.get("endMonth", 0);
            endTime = fields.get("endTime", 0);
            startMonth = fields.get("startMonth", 0);
            startTime = fields.get("startTime", 0);
            startYear = fields.get("startYear", 0);
        }
        if (fields.get("serialVersionOnStream", 0) == 0) {
            if (useDaylight) {
                startMode = endMode = DOW_IN_MONTH_MODE;
                endDay = fields.get("endDay", 0);
                endDayOfWeek = fields.get("endDayOfWeek", 0) - 1;
                startDay = fields.get("startDay", 0);
                startDayOfWeek = fields.get("startDayOfWeek", 0) - 1;
            }
        } else {
            dstSavings = fields.get("dstSavings", 0);
            if (useDaylight) {
                endMode = fields.get("endMode", 0);
                startMode = fields.get("startMode", 0);
                int length = stream.readInt();
                byte[] values = new byte[length];
                stream.readFully(values);
                if (length >= 4) {
                    startDay = values[0];
                    startDayOfWeek = values[1];
                    if (startMode != DOM_MODE) {
                        startDayOfWeek--;
                    }
                    endDay = values[2];
                    endDayOfWeek = values[3];
                    if (endMode != DOM_MODE) {
                        endDayOfWeek--;
                    }
                }
            }
        }
    }

We are not yet that experienced with FST, but it looks like the mismatch between writing always some bytes after writeFields, but it is only reading it under certain conditions. Is this what is causing the problem? If so, how can this be solved?

[RuedigerMoeller]

the reimplementation of android is quite sloppy, they frequently rely on internal details of JDK serialization implementation, so there are quite some cases where FST works for JDK, but fails with android. Which FST version and which Android version did you use (as android N is quite different to prior versions) ?

[simplysoft]

We are using the latest FST version (2.47) and did test it on Marshmallow. We have not yet tested in on android N