Xamarin Forms app crashes when replication started on Android

jda · May 24, 2019, 9:10pm

Hi

Well, the physical device I am testing on is running 8.1 - and is 32 bit (apparently). So I had to enable armeabi-v7a and x86 only to make it work.

… oh. and yes, it seems that armabi is not an option any more - so started by removing that

/John

panosmm · May 24, 2019, 9:44pm

But is it runnng in that device when targeting 9.0 AND enabling arm64?

jda · May 24, 2019, 10:07pm

… No, I cannot enable arm64. So target 9.0 and select the ABIs as I mentioned (only 32 bits).

avia_bdg · May 27, 2019, 7:52am

I have the same problem. When running armeabi-v7a (32 bit) only everything works, but as soon as you add arm64-v8a (64 bit) the app runs until you start replication, then it crashes. I tried the sample TravelSample app as well, and it has the same. Tried with Xamarin Forms 2.5 and 4.0.

Xamarin.Android Version: 9.2.3.0

My test device is Samsung A8 (2018) running Android Pie 9.0.

avia_bdg · May 27, 2019, 8:07am

I tried with the latest Xamarin.Android from Preview channel, Version 9.3.0.19. Same result, replicator crashes on 64-bit.

jda · May 27, 2019, 8:37am

@blake.meike can you (or someone else at Couchbase) help us here.

Is this a Couchbase problem - or a Xamarin problem?

Anyway, it is not isolated to my setup - so could be a more general thing that should be addressed accordingly

avia_bdg · May 27, 2019, 10:43am

I went back in my archives to see which version of the app worked, and when it stopped working. I have a version 2.0.4 from April 1, 2019 which works. I have an update 2.0.5 from April 3, 2019 which doesn’t. It was only a minor change, and not the cause of the problem (I tested it anyway to be sure). No updates to NuGet packages or anything like that.

So if it’s not something related to my project, what else changed?
April 2, 2019: Visual Studio 2019 for Mac version 8.0 (8.0.0.3001)
April 2, 2019: Xamarin.Android 9.2.0.5 in Visual Studio 2019 and Visual Studio 2019 for Mac

I have an old environment set up on an old laptop, with Visual Studio 2017 for Mac 7.8.1 with Xamarin.Android 9.1.8.0 (the immediate prior release version from February 13, 2019). The replication does not crash.

So most likely whatever is making Couchbase Lite replication no longer work on 64-bit ARM is due to something that changed in Xamarin.Android 9.2.0.5 bundled with VS 2019.

avia_bdg · May 27, 2019, 10:57am

Although it didn’t crash at the start of the replication, it did just crash near/at the end of it (replication takes long). Error is slightly different:

[.] * Assertion at /Users/builder/jenkins/workspace/xamarin-android-d15-9/xamarin-android/external/mono/mono/mini/unwind.c:640, condition `cfa_reg != -1’ not met
[libc] Fatal signal 6 (SIGABRT), code -6 (SI_TKILL) in tid 11107 (Thread-21938), pid 18052 (a.app.app)

So now I am again not sure.

avia_bdg · May 27, 2019, 11:49am

I did a sanity check and ran my 2.0.4 binary from April 1, 2019 again. It does complete the replication without crash.

I went back in git to make a new build of 2.0.4 with the old VS 2017 environment. It starts to replicate, but then just… stops. App unresponsive.

[.] [Debug] * REPLICATION PROGRESS: 1569 / 1884
Thread finished: #352
Thread finished: #351
Thread started: #353
[a.app.ap] Thread[3,tid=31363,WaitingInMainSignalCatcherLoop,Thread*=0x7a81816400,peer=0x12cca538,“Signal Catcher”]: reacting to signal 3
[a.app.ap]
[a.app.ap] Wrote stack traces to ‘[tombstoned]’

Uninstall, clean all, tried again. Replication starts. And it completes without error. Uninstall, clean all, tried again. And near the end I get the different error cfa_reg != -1 not met assertion. Perhaps there is something with the data? I tried with a different smaller dataset, same cfa_ref != -1 not met near the end.

If I run my current latest source code (CBL 2.5, XF 4, etc.) on VS 2019 it crashes instantly on replication with fd >= 0 not met, and if I run my current latest source code on VS 2017 it crashes at some point during the replication with cfa_reg != -1 instead.

I tried the TravelSample project on my old VS 2017 in 64-bit. It crashes with fd >= 0 not met.

So my old source code replicates (sometimes) on the old VS 2017. Couchbase Lite 2.1.2. Xamarin Forms 3.5. But updated code will not work on either.

I now disabled arm64-v8a and that build works without any problem on the 64-bit phone. Replication without any error. 64-bit is completely impossible atm with CBL.

Starting August 1, 2021: Google Play will stop serving apps without 64-bit versions on 64-bit capable devices, meaning they will no longer be available in the Play Store on those devices.

There is some time pressure here.

jda · May 27, 2019, 12:45pm

Agree on that!

I don’t think it is a problem with data - as then it wouldn’t work on 32 bit…

I have resigned to running 32 bit until VS2019 is a little more stable - and/or perhaps something at the CBL end can fix it. However, I really hope (=expect) this to be fixed soon as you’re entirely correct about the future requirement for this to work

borrrden · May 27, 2019, 8:42pm

I’ve been out of the office until now but let me clear a few things up:

At least one of those assertion messages (fd >= 0), and most likely the other as well, comes from within the bowels of Xamarin’s .NET runtime and is not a result of anything in particular that Couchbase does. It’s not the first time this problem has come up and I’ve raised it with Xamarin before. It is difficult to get them a reliable reproduction case though. The problem is, if I remember correctly, running out of file descriptors within their thread pool implementation. I did not receive any advice about how to potentially work around it any of the times I looked into it, and I don’t expect there is anything to do this time other than to file a bug report with Microsoft.
There is nothing missing in the documentation in terms of what libraries to install. The Android support library is a dependency of the main library and NuGet should pull it in automatically (it is a little bit better about this with the new PackageReference format rather than the unreliable packages.config method of doing things).
I am in no better of a position to do anything about this than anyone else. You might have noticed that in the event that a stack trace shows up, there is no Couchbase code at all in it. Also to add to the frustration this is not 100% reproducible and is subject to various races I’d imagine.

Has anyone in the thread been talking to Xamarin? Have they responded? If you’d like I can file a bug report but I can only point them to this forum and indicate that this has happened before in previous incarnations of the Xamarin runtime.

jda · May 27, 2019, 9:31pm

Thanks for your response @borrrden

I have not filed a bug with Xamarin - I’m still a little new to Xamarin . However, I did search quite thoroughly following the comments from you guys - but did not find an exact match.

However, it seems to be very easy and consistent to reproduce, e.g. using the TravelSample.

Good to hear that the support libraries should be pulled in by NuGet. I don’t know why they didn’t in my case… Perhaps it is related to upgrading? No, I tried to remove and re-add them, so it must be something else…? Anyway, if that is only a problem in my case then don’t do more about that.

Well, I just did try to remove Couchbase.Lite and Couchbase.Lite.Support.Android - and then re-add them to the Droid project of the TravelSample… - and it did not add the support library … Nor did it add the NewtonSoft.Json nor SimpleInjector packages as specified in the nuspec file you linked to…

Is there something fundamental I’m missing here??

borrrden · May 27, 2019, 9:48pm

If you are still using packages.config then the way NuGet works is a bit insane in my opinion. It will simply look through the deps and add them all as first level deps to the project. After that if you remove them, etc, it won’t retroactively look at any deps that it has broken. If you reinstall, in theory, it should restore the deps that it is missing but I find it to be very hit or miss which is why I do not use packages.config anymore. That goes double for VS for Mac. The new style of dependency management is much more reliable (csproj <PackageReference> entries). Also note if you are using the newer version the deps of the package won’t show up in your csproj file, as instead nuget manages them as it should, by walking the dep chain and installing anything it needs (and then removing anything it doesn’t on uninstall). If you drill down into the deps on VS Windows you will see what has been installed and I believe on VS Mac as well.

As far as the bug in question here is a report someone filed back in 2016 with similar “workaround” steps to the ones in this thread. Here is a post (and bug report) that I filed myself a couple years back that went unanswered.

The bug got closed as not reproducible, so if you were to file another one with a project setup to reproduce the issue that would probably go a long way toward helping Xamarin fix the bug. Or if you give me the exact reproduction steps leaving no detail out (explained as if I were a total dunce and wanted to get a monkey to do it. That’s the best way to not frustrate the person receiving the bug report ) I can file it on your behalf. The repo has over 1,000 open issues so hopefully it doesn’t get lost in the fire hose.

jda · May 28, 2019, 6:08am

Ok, I just took the TravelSample project - and as far as I can see it has the “new” way of doing it, like:

<PackageReference Include="Couchbase.Lite">
   <Version>2.5.0</Version>
</PackageReference>

However, it did not install the support package (nor the two others that was mentioned) - and I had to add at least the support package manually - or the project doesn’t run…

If I understand you correctly then it should just have worked - but it doesn’t. I guess that I should be able to see the dependent packages being installed as well? Or are they somehow “hidden” away from me?

borrrden · May 28, 2019, 7:29am

It does appear that on the VS Mac UI for Android (and iOS) projects that the dependencies are not shown, but they are nonetheless there. I’d consider this a flaw in the Visual Studio product. If you have a .NET Standard project and do the same you will see the more sensible UI as follows (note that you won’t see a support library here since it is a .NET Standard project):

However, even without installing the support library, I confirmed that I was able to call Couchbase.Lite.Support.Droid.Activate(ApplicationContext) (which is an API inside of the support package) so at least NuGet appears to be functioning correctly. I am curious how you got into a position where it does not work…or are you referring to the topic issue of this thread (that it runs, but it crashes)?

jda · May 28, 2019, 7:47am

Hi Jim

Ok, things may have been mixed up in this thread. So let me test that in a clean environment just to figure out if there is an issue or not. It seems that there shouldn’t be an issue - so I really hope that I’m wrong here

jda · May 28, 2019, 7:59am

… and you’re absolutely right!

It works by just adding the Couchbase.Lite package to all projects

So I probably had more things going while trying to fix the original issue… Sorry for confusing things!

avia_bdg · May 28, 2019, 12:41pm

I am not so concerned who is to blame for this, Couchbase, Microsoft, Google. More worried about what happens in August.

To reproduce the issue:

download the TravelSample app, and set it up as per the tutorial (you need a server for replication)
- I set up a Couchbase Community server and Gateway in Windows 10 and configured as per tutorial, confirmed it worked with the sample Python app
- have to remove the “delta updates” flag from the Gateway JSON config file as this is an Enterprise feature and makes Gateway Community refuse to start
- fixed the log file path in the Gateway JSON config file to conform to Windows 10 environment
open the TravelSample project file (sln) under dotnet directory in Visual Studio, I use Visual Studio 2019 for Mac, but 2017/2019 Mac/Windows should not matter
- I changed line 41 of LoginModel.cs in TravelSample.Core to replace “localhost” with the IP of my VM hosting Couchbase
- I changed line 35 of Constants.cs in TravelSample.Core to replace “localhost” with the IP of my VM hosting Couchbase
- Since I used Couchbase Community I removed the Couchbase Lite Enterprise NuGet packages from both projects and replaced them with the Couchbase Lite Community packages.
- left all the other NuGets on the version as they came, updating them makes no difference
open the Options on the TravelSample.Android project, and go to Android Build > Advanced, fix/change to support the ABIs: armeabi-v7a and arm64-v8a.
- on Visual Studio this is project Properties, then Android Options, Advanced, Supported Architectures
run (debug or release) with the TravelSample.Android project as startup project (default) on a 64-bit capable device (in my case Samsung A8 (2018), latest security pack, Android 9), so the arm64-v8a binary is used.
log into the app with a valid user (per tutorial: admin / password; what I used)
will crash as soon as the replication starts

[.] * Assertion at /Users/builder/jenkins/workspace/xamarin-android-d16-0/xamarin-android/external/mono/mono/metadata/threadpool-io.c:368, condition `fd >= 0’ not met

For comparison, when I go back to the Android project, and remove arm64-v8a (so it’s only armeabi-v7a enabled), rebuild the solution, remove the previous app from the phone, run it again, no crash on replication, sample app works as designed.

My setup is Visual Studio 2019 for Mac, with Xamarin.Android 9.2.3.0. Also tried on Visual Studio 2017 for Mac, with Xamarin.Android 9.1.8.0.

I have also tried the same sample steps on Windows 10 with Visual Studio 2019 and Xamarin.Android 9.3.0.22. Same result:
armeabi-v7a only build replication works
arm64-v8a included build on 64-bit phone crashes when replication starts

05-28 13:36:24.862 F/ ( 9749): * Assertion at /Users/builder/jenkins/workspace/xamarin-android-d16-1/xamarin-android/external/mono/mono/metadata/threadpool-io.c:368, condition `fd >= 0’ not met

If you filed a report please let us know. Since Xamarin is open source, have you considered debugging Xamarin.Android to see why it crashes for Couchbase Lite? And if not, for any reason other than the time it would take to set up and do?

avia_bdg · May 28, 2019, 2:59pm

I am filing a bug report at mono. I will try to see if I can place a ready-to-use Couchbase setup VM somewhere for them to reproduce it, in addition to the solution.

jda · May 28, 2019, 3:21pm

If you set up the Docker containers you only have to change the IP addresses. I’ve done that - but the error is consistent across Community and Enterprise editions.