Replicators do not release resources when they are finished. As such for long-running applications, the replicators end up filling memory and causing an application crash.
We declare the replicator
private Replicator remote;
Initialize the replicator
Endpoint targetEndpoint = new URLEndpoint(new URI(getRemoteURL()));
ReplicatorConfiguration replConfig = new ReplicatorConfiguration(database, targetEndpoint);
replConfig.setReplicatorType(ReplicatorConfiguration.ReplicatorType.PUSH_AND_PULL);
replConfig.setAuthenticator(new BasicAuthenticator(getUsername(), getPassword()));
replConfig.setContinuous(false);
remote = new Replicator(replConfig);
remote.start();
For the test case we run the following code every 5 seconds:
remote.stop();
Endpoint targetEndpoint = new URLEndpoint(new URI(getRemoteURL()));
ReplicatorConfiguration replConfig = new ReplicatorConfiguration(database, targetEndpoint);
replConfig.setReplicatorType(ReplicatorConfiguration.ReplicatorType.PUSH_AND_PULL);
replConfig.setAuthenticator(new BasicAuthenticator(getUsername(), getPassword()));
replConfig.setContinuous(false);
remote = new Replicator(replConfig);
remote.start();
After about 15min the following is the thread pool state for the application:
2019-03-30 11:15:21.527 4980-5095/pos D/DeviceManagement: Thread (121):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.528 4980-5095/pos D/DeviceManagement: Thread (122):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.528 4980-5095/pos D/DeviceManagement: Thread (123):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.528 4980-5095/pos D/DeviceManagement: Thread (124):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.529 4980-5095/pos D/DeviceManagement: Thread (125):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.529 4980-5095/pos D/DeviceManagement: Thread (126):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.529 4980-5095/pos D/DeviceManagement: Thread (127):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.529 4980-5095/pos D/DeviceManagement: Thread (128):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.530 4980-5095/pos D/DeviceManagement: Thread (129):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.530 4980-5095/pos D/DeviceManagement: Thread (130):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.530 4980-5095/pos D/DeviceManagement: Thread (131):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.530 4980-5095/pos D/DeviceManagement: Thread (132):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.531 4980-5095/pos D/DeviceManagement: Thread (133):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.531 4980-5095/pos D/DeviceManagement: Thread (134):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.531 4980-5095/pos D/DeviceManagement: Thread (135):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.532 4980-5095/pos D/DeviceManagement: Thread (136):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.532 4980-5095/pos D/DeviceManagement: Thread (137):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.532 4980-5095/pos D/DeviceManagement: Thread (138):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.532 4980-5095/pos D/DeviceManagement: Thread (139):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.533 4980-5095/pos D/DeviceManagement: Thread (140):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.533 4980-5095/pos D/DeviceManagement: Thread (141):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.533 4980-5095/pos D/DeviceManagement: Thread (142):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.533 4980-5095/pos D/DeviceManagement: Thread (143):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.534 4980-5095/pos D/DeviceManagement: Thread (144):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.534 4980-5095/pos D/DeviceManagement: Thread (145):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.534 4980-5095/pos D/DeviceManagement: Thread (146):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.534 4980-5095/pos D/DeviceManagement: Thread (147):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.535 4980-5095/pos D/DeviceManagement: Thread (148):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.535 4980-5095/pos D/DeviceManagement: Thread (149):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.535 4980-5095/pos D/DeviceManagement: Thread (150):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.535 4980-5095/pos D/DeviceManagement: Thread (151):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.536 4980-5095/pos D/DeviceManagement: Thread (152):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.536 4980-5095/pos D/DeviceManagement: Thread (153):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.536 4980-5095/pos D/DeviceManagement: Thread (154):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.537 4980-5095/pos D/DeviceManagement: Thread (155):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.537 4980-5095/pos D/DeviceManagement: Thread (156):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.537 4980-5095/pos D/DeviceManagement: Thread (157):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.537 4980-5095/pos D/DeviceManagement: Thread (158):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.538 4980-5095/pos D/DeviceManagement: Thread (159):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.538 4980-5095/pos D/DeviceManagement: Thread (160):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.538 4980-5095/pos D/DeviceManagement: Thread (161):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.538 4980-5095/pos D/DeviceManagement: Thread (162):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.539 4980-5095/pos D/DeviceManagement: Thread (163):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.539 4980-5095/pos D/DeviceManagement: Thread (164):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.539 4980-5095/pos D/DeviceManagement: Thread (165):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.539 4980-5095/pos D/DeviceManagement: Thread (166):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.540 4980-5095/pos D/DeviceManagement: Thread (167):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.540 4980-5095/pos D/DeviceManagement: Thread (168):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.540 4980-5095/pos D/DeviceManagement: Thread (169):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.541 4980-5095/pos D/DeviceManagement: Thread (170):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.541 4980-5095/pos D/DeviceManagement: Thread (171):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.541 4980-5095/pos D/DeviceManagement: Thread (172):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.541 4980-5095/pos D/DeviceManagement: Thread (173):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.542 4980-5095/pos D/DeviceManagement: Thread (174):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.542 4980-5095/pos D/DeviceManagement: Thread (175):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.542 4980-5095/pos D/DeviceManagement: Thread (176):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.542 4980-5095/pos D/DeviceManagement: Thread (177):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.543 4980-5095/pos D/DeviceManagement: Thread (178):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.543 4980-5095/pos D/DeviceManagement: Thread (179):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.543 4980-5095/pos D/DeviceManagement: Thread (180):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.543 4980-5095/pos D/DeviceManagement: Thread (181):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.544 4980-5095/pos D/DeviceManagement: Thread (182):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.544 4980-5095/pos D/DeviceManagement: Thread (183):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.544 4980-5095/pos D/DeviceManagement: Thread (184):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.545 4980-5095/pos D/DeviceManagement: Thread (185):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.545 4980-5095/pos D/DeviceManagement: Thread (186):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.545 4980-5095/pos D/DeviceManagement: Thread (187):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.545 4980-5095/pos D/DeviceManagement: Thread (188):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.546 4980-5095/pos D/DeviceManagement: Thread (189):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.546 4980-5095/pos D/DeviceManagement: Thread (190):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.546 4980-5095/pos D/DeviceManagement: Thread (191):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.547 4980-5095/pos D/DeviceManagement: Thread (192):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.547 4980-5095/pos D/DeviceManagement: Thread (193):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.547 4980-5095/pos D/DeviceManagement: Thread (194):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.547 4980-5095/pos D/DeviceManagement: Thread (195):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.548 4980-5095/pos D/DeviceManagement: Thread (196):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.548 4980-5095/pos D/DeviceManagement: Thread (197):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.548 4980-5095/pos D/DeviceManagement: Thread (198):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.548 4980-5095/pos D/DeviceManagement: Thread (199):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.548 4980-5095/pos D/DeviceManagement: Thread (200):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.549 4980-5095/pos D/DeviceManagement: Thread (201):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.549 4980-5095/pos D/DeviceManagement: Thread (202):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.549 4980-5095/pos D/DeviceManagement: Thread (203):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.549 4980-5095/pos D/DeviceManagement: Thread (204):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.549 4980-5095/pos D/DeviceManagement: Thread (205):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.550 4980-5095/pos D/DeviceManagement: Thread (206):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.550 4980-5095/pos D/DeviceManagement: Thread (207):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.550 4980-5095/pos D/DeviceManagement: Thread (208):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.550 4980-5095/pos D/DeviceManagement: Thread (209):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.550 4980-5095/pos D/DeviceManagement: Thread (210):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.551 4980-5095/pos D/DeviceManagement: Thread (211):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.551 4980-5095/pos D/DeviceManagement: Thread (212):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.551 4980-5095/pos D/DeviceManagement: Thread (213):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.551 4980-5095/pos D/DeviceManagement: Thread (214):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.551 4980-5095/pos D/DeviceManagement: Thread (215):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.552 4980-5095/pos D/DeviceManagement: Thread (216):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.552 4980-5095/pos D/DeviceManagement: Thread (217):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.552 4980-5095/pos D/DeviceManagement: Thread (218):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.552 4980-5095/pos D/DeviceManagement: Thread (219):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.552 4980-5095/pos D/DeviceManagement: Thread (220):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.553 4980-5095/pos D/DeviceManagement: Thread (221):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.553 4980-5095/pos D/DeviceManagement: Thread (222):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.553 4980-5095/pos D/DeviceManagement: Thread (223):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.553 4980-5095/pos D/DeviceManagement: Thread (224):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.554 4980-5095/pos D/DeviceManagement: Thread (225):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.554 4980-5095/pos D/DeviceManagement: Thread (226):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.554 4980-5095/pos D/DeviceManagement: Thread (227):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.554 4980-5095/pos D/DeviceManagement: Thread (228):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.554 4980-5095/pos D/DeviceManagement: Thread (229):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.555 4980-5095/pos D/DeviceManagement: Thread (230):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.555 4980-5095/pos D/DeviceManagement: Thread (231):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.555 4980-5095/pos D/DeviceManagement: Thread (232):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.555 4980-5095/pos D/DeviceManagement: Thread (233):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.555 4980-5095/pos D/DeviceManagement: Thread (234):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.556 4980-5095/pos D/DeviceManagement: Thread (235):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.556 4980-5095/pos D/DeviceManagement: Thread (236):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.556 4980-5095/pos D/DeviceManagement: Thread (237):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.556 4980-5095/pos D/DeviceManagement: Thread (238):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.557 4980-5095/pos D/DeviceManagement: Thread (239):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.557 4980-5095/pos D/DeviceManagement: Thread (240):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.557 4980-5095/pos D/DeviceManagement: Thread (241):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.557 4980-5095/pos D/DeviceManagement: Thread (242):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.557 4980-5095/pos D/DeviceManagement: Thread (243):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.558 4980-5095/pos D/DeviceManagement: Thread (244):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.558 4980-5095/pos D/DeviceManagement: Thread (245):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.558 4980-5095/pos D/DeviceManagement: Thread (246):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.558 4980-5095/pos D/DeviceManagement: Thread (247):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.558 4980-5095/pos D/DeviceManagement: Thread (248):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.559 4980-5095/pos D/DeviceManagement: Thread (249):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.559 4980-5095/pos D/DeviceManagement: Thread (250):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.559 4980-5095/pos D/DeviceManagement: Thread (251):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.559 4980-5095/pos D/DeviceManagement: Thread (252):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.559 4980-5095/pos D/DeviceManagement: Thread (253):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.560 4980-5095/pos D/DeviceManagement: Thread (254):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.560 4980-5095/pos D/DeviceManagement: Thread (255):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.560 4980-5095/pos D/DeviceManagement: Thread (256):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.560 4980-5095/pos D/DeviceManagement: Thread (257):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.560 4980-5095/pos D/DeviceManagement: Thread (258):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.561 4980-5095/pos D/DeviceManagement: Thread (259):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.561 4980-5095/pos D/DeviceManagement: Thread (260):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.561 4980-5095/pos D/DeviceManagement: Thread (261):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.561 4980-5095/pos D/DeviceManagement: Thread (262):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.562 4980-5095/pos D/DeviceManagement: Thread (263):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.562 4980-5095/pos D/DeviceManagement: Thread (264):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.562 4980-5095/pos D/DeviceManagement: Thread (265):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.562 4980-5095/pos D/DeviceManagement: Thread (266):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.562 4980-5095/pos D/DeviceManagement: Thread (267):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.563 4980-5095/pos D/DeviceManagement: Thread (268):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.563 4980-5095/pos D/DeviceManagement: Thread (269):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.563 4980-5095/pos D/DeviceManagement: Thread (270):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.563 4980-5095/pos D/DeviceManagement: Thread (271):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.563 4980-5095/pos D/DeviceManagement: Thread (272):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.564 4980-5095/pos D/DeviceManagement: Thread (273):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.564 4980-5095/pos D/DeviceManagement: Thread (274):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.564 4980-5095/pos D/DeviceManagement: Thread (275):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.564 4980-5095/pos D/DeviceManagement: Thread (276):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.564 4980-5095/pos D/DeviceManagement: Thread (277):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.565 4980-5095/pos D/DeviceManagement: Thread (278):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.565 4980-5095/pos D/DeviceManagement: Thread (279):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.565 4980-5095/pos D/DeviceManagement: Thread (280):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.565 4980-5095/pos D/DeviceManagement: Thread (281):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.565 4980-5095/pos D/DeviceManagement: Thread (282):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.566 4980-5095/pos D/DeviceManagement: Thread (283):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.566 4980-5095/pos D/DeviceManagement: Thread (284):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.566 4980-5095/pos D/DeviceManagement: Thread (285):Thread[OkHttp Dispatcher,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.566 4980-5095/pos D/DeviceManagement: Thread (286):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.567 4980-5095/pos D/DeviceManagement: Thread (287):Thread[OkHttp Dispatcher,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.567 4980-5095/pos D/DeviceManagement: Thread (288):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.567 4980-5095/pos D/DeviceManagement: Thread (289):Thread[OkHttp Dispatcher,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.567 4980-5095/pos D/DeviceManagement: Thread (290):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.568 4980-5095/pos D/DeviceManagement: Thread (291):Thread[OkHttp Dispatcher,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.568 4980-5095/pos D/DeviceManagement: Thread (292):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.568 4980-5095/pos D/DeviceManagement: Thread (293):Thread[OkHttp Dispatcher,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.568 4980-5095/pos D/DeviceManagement: Thread (294):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.568 4980-5095/pos D/DeviceManagement: Thread (295):Thread[OkHttp Dispatcher,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.569 4980-5095/pos D/DeviceManagement: Thread (296):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.569 4980-5095/pos D/DeviceManagement: Thread (297):Thread[OkHttp Dispatcher,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.569 4980-5095/pos D/DeviceManagement: Thread (298):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.569 4980-5095/pos D/DeviceManagement: Thread (299):Thread[OkHttp Dispatcher,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.569 4980-5095/pos D/DeviceManagement: Thread (300):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.570 4980-5095/pos D/DeviceManagement: Thread (301):Thread[OkHttp Dispatcher,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.570 4980-5095/pos D/DeviceManagement: Thread (302):Thread[AsyncTask #32,5,main]:state:TIMED_WAITING
2019-03-30 11:15:21.570 4980-5095/pos D/DeviceManagement: Thread (303):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.570 4980-5095/pos D/DeviceManagement: Thread (304):Thread[OkHttp Dispatcher,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.570 4980-5095/pos D/DeviceManagement: Thread (305):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.571 4980-5095/pos D/DeviceManagement: Thread (306):Thread[OkHttp Dispatcher,4,main]:state:TIMED_WAITING
2019-03-30 11:15:21.571 4980-5095/pos D/DeviceManagement: Thread (307):Thread[ReplicatorListenerThread,5,main]:state:WAITING
2019-03-30 11:15:21.571 4980-5095/pos D/DeviceManagement: Thread (308):Thread[OkHttp http://…,4,main]:state:RUNNABLE
2019-03-30 11:15:21.571 4980-5095/pos D/DeviceManagement: Thread (309):Thread[OkHttp WebSocket http://…,4,main]:state:WAITING
2019-03-30 11:15:21.572 4980-5095/pos D/DeviceManagement: Thread pool size:309
There appear to be two issues here: 1) The ReplicatorListenerThreads are never released and 2) OkHttp Connections/ConnectionPools are not being managed correctly.
For the next test case we run the following code every 5 seconds instead of the previous cocde
remote.stop();
remote.start();
After about 15min the following is the thread pool state for the application:
2019-03-30 11:27:51.474 6252-6360/pos D/DeviceManagement: Thread (59):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.475 6252-6360/pos D/DeviceManagement: Thread (60):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.475 6252-6360/pos D/DeviceManagement: Thread (61):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.475 6252-6360/pos D/DeviceManagement: Thread (62):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.476 6252-6360/pos D/DeviceManagement: Thread (63):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.476 6252-6360/pos D/DeviceManagement: Thread (64):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.476 6252-6360/pos D/DeviceManagement: Thread (65):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.477 6252-6360/pos D/DeviceManagement: Thread (66):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.477 6252-6360/pos D/DeviceManagement: Thread (67):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.477 6252-6360/pos D/DeviceManagement: Thread (68):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.477 6252-6360/pos D/DeviceManagement: Thread (69):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.478 6252-6360/pos D/DeviceManagement: Thread (70):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.478 6252-6360/pos D/DeviceManagement: Thread (71):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.478 6252-6360/pos D/DeviceManagement: Thread (72):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.478 6252-6360/pos D/DeviceManagement: Thread (73):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.479 6252-6360/pos D/DeviceManagement: Thread (74):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.479 6252-6360/pos D/DeviceManagement: Thread (75):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.479 6252-6360/pos D/DeviceManagement: Thread (76):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.479 6252-6360/pos D/DeviceManagement: Thread (77):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.480 6252-6360/pos D/DeviceManagement: Thread (78):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.480 6252-6360/pos D/DeviceManagement: Thread (79):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.480 6252-6360/pos D/DeviceManagement: Thread (80):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.480 6252-6360/pos D/DeviceManagement: Thread (81):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.481 6252-6360/pos D/DeviceManagement: Thread (82):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.481 6252-6360/pos D/DeviceManagement: Thread (83):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.481 6252-6360/pos D/DeviceManagement: Thread (84):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.482 6252-6360/pos D/DeviceManagement: Thread (85):Thread[OkHttp Dispatcher,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.482 6252-6360/pos D/DeviceManagement: Thread (86):Thread[OkHttp ConnectionPool,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.482 6252-6360/pos D/DeviceManagement: Thread (87):Thread[OkHttp Dispatcher,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.482 6252-6360/pos D/DeviceManagement: Thread (88):Thread[OkHttp Dispatcher,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.483 6252-6360/pos D/DeviceManagement: Thread (89):Thread[OkHttp Dispatcher,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.483 6252-6360/pos D/DeviceManagement: Thread (90):Thread[OkHttp Dispatcher,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.483 6252-6360/pos D/DeviceManagement: Thread (91):Thread[AsyncTask #13,5,main]:state:TIMED_WAITING
2019-03-30 11:27:51.483 6252-6360/pos D/DeviceManagement: Thread (92):Thread[OkHttp Dispatcher,4,main]:state:TIMED_WAITING
2019-03-30 11:27:51.484 6252-6360/pos D/DeviceManagement: Thread pool size:92
This shows that the ReplicatorListenerThread issue is tied to creating new Replicator objects, but the OkHttp issue is tied to the “start()” function call on a Replicator object.