Online Variant of Parcel Allocation in Last-mile Delivery

Online Variant of Parcel Allocation in Last-mile Delivery

Yuan Liang 1State Key Laboratory of Software Development Environment, School of Computer Science, Beihang University 1    Xiuguo Bao 2National Computer Network Emergency Response Technical Team/Coordination Center of China2
Abstract

We investigate the problem of last-mile delivery, where a large pool of citizen crowd-workers are hired to perform a variety of location-specific urban logistics parcel delivering tasks. Current approaches focus on offline scenarios, where all the spatio temporal information of parcels and workers are given. However, the offline scenarios can be impractical since parcels and workers appear dynamically in real applications, and their information is not known in advance. In this paper, in order to solve the shortcomings of the offline setting, we first formalize the online parcel allocation in last-mile delivery problem, in which all parcels were put in pop-stations in advance, while workers arrive dynamically. Then we propose an algorithm which provides theoretical guarantee for the parcel allocation in last-mile delivery. Finally, we verify the effectiveness and efficiency of the proposed method through extensive experiments on real and synthetic datasets.

1 Introduction

Last-mile delivery in urban logistics, where citizen volunteers are incentivized to deliver location-specific parcels, has recently attracted strong commercial interest [4, 17]. In real-life scenario, companies take the goods to the high-capacity warehouses (pop-stations), then the consumers will be notified and collect their parcels via mobile applications. If a parcel is not collected within 3 days, it is considered as a failed delivery.

Therefore, in order to deal with unattended parcels, we utilize the power of crowd workers to enhance the last-mile delivery. In particular, crowd workers can take parcels from pop-stations to consumers. For example, the Yongjia of Beihang University is a pop-station, and many parcels will be collected from the pop-station, consumers will be noticed to collect their parcels by mobile applications. However, many parcels can not be delivered to consumers as consumers are not present when deliveries are made. This situation can be dealt with if there are some crowd-workers who can help consumers to collect parcels at pop-stations and deliver them to consumers. Recall the example of Beihang University, each student can become a crowd worker and collect parcels from Yongjia, and delivery them to consumers. As shown in Fig. 1, all parcels will be put in pop-stations, denoted by , and all workers are denoted by , parcels will be delivered to consumers by crowd-workers.

Figure 1: The pattern of assign parcels to workers

In this paper, we mainly discuss the problem of online parcel allocation in last-mile delivery, in which all parcels are put in the pop-station in advance, while crowd-workers use different time period to collect parcels, and each worker has different schedule that they will arrive dynamically. And crowd workers are rewarded with a certain amount of money according to their speed and the distance of the parcels. In addition, each worker has different working hours to collect different parcels within a day. For example, Alice and Bob are crowd workers in a community, and there is a pop-station in their community, Alice use part-time to collect parcels from the pop-station to customers, and Bob is a full-time crowd worker. Therefore, crowdsourcing platform in order to pursuit the maximum benefit that they will give a reasonable parcel allocation.

As introduced in [17], under offline scenarios, the parcel allocation problem in last-mile delivery can be solved by being reduced to the problem of min-cost flow model [1], where the source node is the only surplus and the sink node is the only demand node, and the remaining nodes represent parcels and crowd-workers, which serve as transshipment nodes. However, the reduction can not be performed under online scenarios, the offline solution becomes infeasible since the arrival orders of workers are unknown in dynamic environments. Therefore, it is necessary for us to propose an algorithm to solve online parcel allocation problem in last-mile delivery. To further illustrate this motivation, we go through a toy example as follows.

Example 1: Suppose in a social community, we have 8 parcels and 4 workers on a crowd-tasking platform, and each worker has a capacity, which is the maximum number of tasks that can be assigned to him/her. In this example, have capacities of 2, 4, 3 and 2, respectively (in brackets). Table 1 presents the utility (i.e. money) values between each pair of task and worker, which not only depends on the weight and distance of parcels also depends on the speed workers and the means of transportion of workers. Under the offline scenarios, we are able to derive all information of the parcels and workers, and the allocation is , ,, , , , , and . However, in dynamic online scenarios, the information of workers is not known in advance, and the allocation heavily depends on the order of crowd-workers. Suppose there are two orders of the crowd-workers, the first order is , and the second order is . For the first order, when arrives, will collect ; when arrives, will collect ; when arrives, will collect ; when arrives, will collect . For the second order, when arrives, will collect ; when arrives, will collect ; when arrives, will collect ; finally, arrives, will collect . Therefore, the online setting heavily depends on the order of crowd-workers.

0.9 0.4 0.5 0.9 0.4 0.8 0.3 0.9
0.2 0.2 0.6 0.3 0.2 0.9 0.8 0.3
0.4 0.5 0.2 0.4 0.7 0.2 0.2 0.7
0.3 0.6 0.4 0.6 0.9 0.4 0.9 0.2
Table 1: Utility between parcels and workers

To the best of our knowledge, this is the first work that studies the online scenario in last-mile delivery problem. It is therefore crucial to design an efficient and effective on-line algorithm dedicated for the last-mile delivery. In this paper, we make the following contributions:

a). We identify a new online scenario of parcel allocation in last-mile delivery model.

b). We propose an algorithm with slight power, whose competitive ratio is , where .

c). We conduct extensive experiments on real and synthetic datasets to evaluate the efficiency and effectiveness of our proposed algorithms.

The rest of the paper is organized as follows. We present the problem definition in Section 2. In Section 3, we review related works. The algorithms of our problem and the theoretical analysis is proposed in Section 4. We conduct experiments in Section 5 to evaluate the performance of our proposed solutions. The conclusion of our work is given in Section 6.

2 Problem Statement

We first introduce several concepts and then formally define the dynamic online scenario in last-mile delivery.

Definition 1 (Parcel): All parcels are located in pop-stations, and there are parcels, denoted by . Each parcel has a utility for each worker, denoted by .

Definition 2 (Worker): A crowd worker (”worker” for short), denoted by , where represents the time of worker collect parcels within a day. In addition, capacity is the maximum number of parcels that worker intends to collect, and denotes the time of worker delivery parcel . And there are workers, denoted by .

We then define the utility value that parcels are allocated workers as follows.

Definition 3 (Utility value): The utility value that a worker perform a parcel is measured by .

where denotes the utility of parcel is collected by worker . Let denote parcel is assigned to worker and for the opposite case. Our goal is to maximize that

(1)

subject to

(2)
(3)
(4)

Constraint (2) means that each parcel must be assigned to no more than one worker. Constraint (3) restricts a worker has a capacity, which is the maximum number of parcels that can be assigned to him/her. Constraint (4) means that each worker has different working hours in a day (i.e. some workers are part-time, and some workers are full-time), where denotes the time of worker collect parcel , and represents the working hours of worker within a day.

The online algorithm has another constraint that once a parcel is allocated to a worker , the allocation of cannot be changed. And the performance of online algorithm is usually compared with the optimal allocation of the offline scenario and heavily depends on the arriving orders of crowd-workers. Moreover, we evaluate the online algorithm using the notion of competitive ratio, which is a lower bound on the ratio between the utility of the algorithm and the utility of the optimal offline algorithm (all information of all parcels and workers are known in advance). For example, a competitive ratio of 1/2 would imply that an algorithm always achieves a utility that is at least half as good as optimal. We say an algorithm is if its competitive ratio is at least .

The optimal solution of the offline problem is introduced as follows.

We introduce an optimal offline parcel allocation in last-mile delivery, we can reduce our offline problem to the min-cost flow problem. We first construct a flow network , where , is a source node and is a sink node. In flow network, there are three types of arcs, denoted by , and respectively, and the network is shown in Fig. 2. The first type arcs represents the source nodes to parcels, with capacity 1 and the cost of this type arcs is 0; the second type arcs is from parcels to workers, with capacity 1 and the cost of this type arcs is , where , ; The third type arcs is from workers to sink node, with capacity is the maximum number parcels of worker can collect. Then, we use existing flow algorithms to obtain the optimal offline parcel allocation in last-mile delivery, e.g., simplex algorithm [9], to calculate the optimal utility value.

Figure 2: Min-cost flow model

3 Related Work

In this section, we review related works from two categories, spatial crowdsourcing and last-mile delivery.

3.1 Mobile crowdsourcing

In recent years, with the development of mobile Internet and distributed systems [16, 15], more and more applications of mobile crowdsourcing are emerging, e.g. Uber, Gigwalk, etc. Particularly, task assignment is one of the major topics on mobile crowdsourcing. [20, 5] introduced task assignment problem in offline setting by learning the quality of crowd workers.

In addition, there have been several researches on online allocation, such as [11, 14, 13]. [14] propose a general bipartite-matching-based framework to address the dynamic task allocation in online spatial crowdsourcing platforms. Moreover, [7] studies online task assignment, where tasks arrival while workers are not dynamic. And[18] introduced that minimize the total waiting time before the crowd workers arrives at a specific location of tasks. Although the aforementioned works study the various problem on online allocation, most of them do not address the last-mile delivery.

3.2 Last-mile delivery

In recent years, there have been several researches in the last-mile delivery[17, 4]. [17] introduced the planning and scheduling parcels so as to minimize the additional travelling cost. They propose an effective large-scale mobile crowd-tasking model in which a number workers deliver a variety of location-specific urban logistics parcels. However, they do not address the parcel allocation problem for workers that arrive dynamically, and are thus impractical for dynamic environments.

In addition, there are many approaches which focus on a particular class of the workers’ utility. For example, [8] introduced a centralized allocation algorithm that maximize allocated tasks, while satisfying a set of constraints. [6] consider that tasks with different requirements of quality of sensing are typically associated with specific locations and mobile users are constrained by time budgets. Moreover, [4] study the task allocation as an optimization problem with the objective function that maximizing the total utility from all assigned tasks. Although these works study the task allocation problem on mobile crowdsourcing, they rarely consider the challenge of the last-mile delivery and dynamic scenario.

4 Solution

In this section, we propose a baseline algorithm and a primal-dual algorithm to solve the parcel allocation in last-mile delivery. In our problem, all parcels were taken to pop-station, when a crowd-worker arrives, s/he can carry at most a set of parcels . And when a worker arrives, parcels will be allocated to the worker immediately, and the allocation can not be changed.

4.1 Baseline Algorithm

In this subsection, we present a greedy as baseline algorithm. The main idea of greedy is to set all parcels in pop-station, and crowd-workers arrive dy dynamically. Note that when a crowd-worker arrives, s/he would carry at most parcels once. In particular, each crowd-worker will select the parcels with the highest utility that satisfies all constraints.

More specifically, let is the number of workers, is the number of parcels, and is the allocation. When a crowd-worker arrives, s/he has a capacity constraint . Let represents the set of parcels of worker will collect. When worker arrives, s/he will repeat this process until there are no any parcels and thereby completes the allocation.

1:
2:A feasible allocation
3: for all .
4:for each parcel  do
5:     buile a list sorted in ascending order of for all workers
6:end for
7:for each new arrival worker  do
8:     Let denote the set of parcels are not allocated
9:     if  then
10:         the worker collect a set of parcels with highest utility that satisfies all constraints
11:         
12:     else
13:         break;
14:     end if
15:end for
16:return the final allocation and the total utility .
Algorithm 1 Greedy

Algorithm 1 illustrates the procedure. Line 1 initializes for each worker. Here, denotes worker collect a set of parcels at most . In lines 2-4, we build a list sorted in ascending order of for all workers such that when a worker arrives, parcels will quickly be allocated to the worker. In lines 5-13, we iteratively process each new arrival worker. Particularly, we adopt a greedy strategy on the arrival worker. In addition, let denotes the set of parcels are not allocated in line 6. In lines 7-12, if , the arrival worker will collect a set of parcels with highest utility that satisfies all constraints. The capacity of arrival worker at most . Finally, we will obtain the maximum utility and the final allocation accordingly in line 14.

Example 1

Here is the process of running Greedy algorithm on Example 1. First, we set the utility of workers and parcels in TABLE 1. Suppose the order of workers is , when worker arrives, s/he will collect parcels that satisfies her/his capacity and obtain the maximum utility. When worker arrives, s/he will collect , the utility of worker is . When worker arrives, s/he will collect that satisfies constraints. When arrives, there is only a parcel is not collected, and will collect , the utility of is 0.9. Finally, the online parcel allocation is .

Complexity analysis. In lines 2-4, we build an inverted list for each parcel , the elements in the list are in form of , and the time complexity of Greedy is . For each of the new arrival worker, the time complexity of Greedy is . Therefore, the time complexity of Greedy is .

4.2 Primal-dual Algorithm

Inspired by online primal-dual algorithm [3, 2, 12], we propose a primal-dual algorithm to solve parcel allocation problem. We first give the duality of the problem, and the standard primal-dual linear programming formulation of our problem are shown as follows.

(5)

subject to

(6)
(7)
(8)

According to [10], and the derivations of its dual are shown as follows.

Step 1. Rewrite the objective (5) as a minimization.

(9)

Step 2. Rewrite each inequality constraint as a ”less than or equal”, and rearrange each constraint so that the right-hand side is 0.

After this step our linear program now looks as follows.

(10)

subject to

(11)
(12)
(13)

Step 3. Define a non-negative dual variable for each inequality constraint. To constraint (12) and (13) we associate variable , and to constraint (11), we associate variable .

Step 4. For each constraint, eliminate the constraint and add , to the objective. Concretely, for the constraints (12) and (13) we would remove it and add the following term to the objective.

(14)

If we do this for each constraint, and maximize the result over the dual variables, we get

(15)

Step 5. Rewrite the objective. And we get

(16)

Step 6. Remove and with constraints.

Step 7. Finally, rewrite the result of the step 6 as a minimization. And the dual is shown as follows.

(17)

subject to

(18)
1:initially
2:for Upon arrival of a new worker allocate a set of parcels to the worker that maximizes  do
3:     if  then
4:         Calculate the utility of worker and its remaining time and set
5:         
6:         
7:     else
8:         break;
9:     end if
10:end for
11:return the final allocation and the total utility .
Algorithm 2 Primal-dual

First, we tempting to design a online primal-dual algorithm (i.e., in Algorithm 2) that achieves a constant worst-case competitive ratio, then we found that no algorithm can be constant-competitive via the following theorem.

Theorem 4.1

Suppose , consider instances of our problem when . Then worst-case competitive-ratio of last-mile delivery on these instances is .

Proof

Suppose there are a set of parcels in the pop-station, and let parcels, for . The parcels partitioned into sets, denoted , and the set has parcels. When a worker arrives, s/he will collect parcels. When algorithm runs on this input, let

(19)

Note that due to feasibility of the algorithm, . Now, run the algorithm on the input with the parameter . Obviously, the output of the algorithm is feasible. To show the competitive ratio, partition the allocation set into such that

(20)

Furthermore, suppose is the algorithm’s allocation and is the optimal offline allocation of the last-mile delivery. Hence, . Now, from , we have

(21)

Let , and . And we have

(22)

To complete the proof of Theorem 1, note that . Here we get

(23)

Therefore, we found that no algorithm can be constant-competitive, and we obtain logarithmically competitive ratio on a certain parameter of our problem through primal-dual algorithm.

5 Experimental Evaluation

5.1 Experiment Setup

In this subsection, we evaluate our proposed algorithms. We use both real and synthetic datasets for experiments.

Datasets. We use the GeoLife dataset from [19] as the real dataset. In the GeoLife dataset, there are 182 users, and we generate the number of parcels 200,400,600,800,1000. For the synthetic data, we generate the working hours of workers following normal distributions, the capacity of workers following uniform distributions. And the utility (i.e., money) of parcels for each worker are generated following (10,20) randomly. The statistics and configurations of the synthetic data are illustrated in Table 2. Default settings are denoted in bold font.

Factor setting
100,200, 300, 400, 500
20, 40, 60, 80, 100
[1,6]
Scalability ()
Table 2: Synthetic dataset

We evaluated our algorithms in terms of allocation utility, running time and memory cost and study the effects of varying the parameters on the performance of the algorithms. The offline scenario is solved by simplex algorithm through min-cost flow model. And the synthetic datasets are created in Python, and all algorithms are implemented in C++ and executed under the Linux Ubuntu operating system. The experiments are conducted on a computer with an Intel Xeon E5620 with a 2.40 GHz 16-core CPU and 12 GB of memory.

5.2 Experiment Results

In this section, we evaluate the proposed algorithms in terms of allocation utility, running time and memory cost. We test the performances of the proposed algorithms by varying the parameters as follows: the size of , the size of , the capacity of worker , and the working hours of workers .

(a) Utility of varying
(b) Time of varying
(c) Memory of varying
(d) Utility of varying
(e) Time of varying
(f) Memory of varying
(g) Utility of varying
(h) Time of varying
(i) Memory of varying
Figure 3: Results of varying , , and

Effect of . We first present the effects of varying , where is set to {20, 40, 60, 80, 100}, and the number of parcels is 200. Figs. (a)a to (c)c show the allocation utility, running time and memory cost, respectively. We have made the following observations. First, the allocation utility increases as increases. This is because when increases, the utility will be calculated among more workers. Second, the running time increases as varies, because as increases, algorithm will consume more time as the size of worker increases. Third, the memory usage increases as increases, which is natural, because the data becomes larger.

Effect of . Next, we set the number of parcels as {100, 200, 300, 400, 500}, and the the number of workers as 40. Figs. (d)d to (f)f show the allocation utility, running time and memory cost, respectively. We have made the following observations. First, the allocation utility increases when increases, because more parcels will be collected by workers. Second, the running time increases as increases. This is because when is larger and is fixed, one worker will take more parcels which results in more calculation time. Third, the memory cost increases as increases, because as increases, it requires more memory.

Effect of capacity . We set the number of parcels as 200, the number of workers as 40, and the capacity of workers as 1 to 6 random numbers, and we present the results of the allocation utility, running time and memory cost in Figs. (g)g to (i)i, respectively, when varying the capacity of the workers. As is shown, the allocation utility increase as increases, because an increase in workers will collect more parcels. Running time and memory cost decrease as worker’s capacity increases.

(a) Utility of varying
(b) Time of varying
(c) Memory of varying
(d) Utility of varying
(e) Time of varying
(f) Memory of varying
(g) Utility on real datasets
(h) Time on real datasets
(i) Memory on real dataset
Figure 4: Results varying working hours and real datasets.

Effect of working hours . We set the number of parcels as 200, the number of workers as 40, and the working hours of workers following normal distribution , and we present the results of the allocation utility, running time and memory cost in Figs. (a)a to (c)c, respectively, when varying the working hours of workers. As is shown, the allocation utility increases as increases since workers will have more time to collect parcels; running time and memory costs have not change much as increases since the number of workers and the number of parcel are fixed. In addition, we set the number of workers and the number of parcels, similar to above. And we set the working hours of workers following normal distribution . Then we present the results of the allocation utility, running time and memory cost in Figs. (d)d to (f)f, respectively. As are shown, the allocation utility increase as increases, because an increase in requires more calculations. Running time decreases when varying the of working hours since parcel allocation will need less worker in one day; memory cost has not change.

Real dataset. Figs. (g)g to (i)i show the results of the allocation utility, running time and memory cost, respectively, on the real dataset [19]. The results on this real dataset present patterns similar to the results with synthetic data.

(a) Utility of scalability test
(b) Time of scalability test
(c) Memory of scalability test
Figure 5: Results on scalability test.

Scalability. Finally, we study the scalability of all the proposed algorithms. Specifically, we set to 200 and to , , , , and . The results are shown in Figs. (a)a to (c)c in terms of allocation utility, running time and memory cost, respectively. We can observe that the allocation utility, running time, and memory cost of all algorithms grow linearly with the size of the data. In addition, the result shows that all the algorithms are scalable in terms of both time and memory cost.

Summary. The Greedy and Primal-dual algorithms are efficient for parcel allocation in last-mile delivery. The Greedy algorithm performs better than Primal-dual in allocation utility, and it is more efficient with regard to running time in more cases since Greedy do not need more time to calculate the value of dual variants.

6 Conclusion

In this paper, in order to solve online parcel allocation problem in last-mile delivery, we first identify a model for online parcel allocation in last-mile delivery. Then, we propose a baseline algorithm to solve this problem. We also present a primal-dual algorithm whose competitive ratio depends logarithmically on a certain parameter of the problem instance. Finally, we verify the effectiveness and efficiency of the proposed solutions through extensive experiments on both real and synthetic datasets. The experiments show promising results for our proposed algorithms.

References

  • [1] R. K. Ahuja, T. L. Magnanti, and J. B. Orlin (1993) Network flows: theory, algorithms, and applications. Cited by: §1.
  • [2] N. Buchbinder and J. Naor (2005) Online primal-dual algorithms for covering and packing problems. In European Symposium on Algorithms, pp. 689–701. Cited by: §4.2.
  • [3] N. Buchbinder and J. Seffi) Naor (2009) The design of competitive online algorithms via a primal: dual approach. Foundations & Trends® in Theoretical Computer Science 3 (2–3), pp. 93–263. Cited by: §4.2.
  • [4] C. Chen, S. Cheng, A. Gunawan, A. Misra, K. Dasgupta, and D. Chander (2014) Traccs: a framework for trajectory-aware coordinated urban crowd-sourcing. In Proceedings of the Second AAAI Conference on Human Computation and Crowdsourcing, Vol. , , pp. 30–40. Note: External Links: Document Cited by: §1, §3.2, §3.2.
  • [5] J. Fan, G. Li, B. C. Ooi, K. Tan, and J. Feng (2015) Icrowd: an adaptive crowdsourcing framework. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, pp. 1015–1030. Cited by: §3.1.
  • [6] S. He, D. Shin, J. Zhang, and J. Chen (2014) Toward optimal allocation of location dependent tasks in crowdsensing. In INFOCOM, 2014 Proceedings IEEE, pp. 745–753. Cited by: §3.2.
  • [7] C. Ho and J. W. Vaughan (2012) Online task assignment in crowdsourcing markets.. In AAAI, Vol. 12, pp. 45–51. Cited by: §3.1.
  • [8] L. Kazemi and C. Shahabi (2012) Geocrowd: enabling query answering with spatial crowdsourcing. In Proceedings of the 20th International Conference on Advances in Geographic Information Systems, Vol. , , pp. 189–198. Note: External Links: Document Cited by: §3.2.
  • [9] Z. Király and P. Kovács (2012) Efficient implementations of minimum-cost flow algorithms. arXiv preprint arXiv:1207.6381. Cited by: §2.
  • [10] S. Lahaie (2008) How to take the dual of a linear program. Columbia University, New York. Cited by: §4.2.
  • [11] A. Mehta (2012) Online matching and ad allocation. Theoretical Computer Science 8 (4), pp. 265–368. External Links: Document Cited by: §3.1.
  • [12] R. Niazadeh and R. D. Kleinberg (2013) A unified approach to online allocation algorithms via randomized dual fitting. arXiv preprint arXiv:1308.5444. Cited by: §4.2.
  • [13] Y. Tong, J. She, B. Ding, L. Chen, T. Wo, and K. Xu (2016) Online minimum matching in real-time spatial data: experiments and analysis. Proceedings of the VLDB Endowment 9 (12), pp. 1053–1064. External Links: Document Cited by: §3.1.
  • [14] Y. Tong, J. She, B. Ding, L. Wang, and L. Chen (2016) Online mobile micro-task allocation in spatial crowdsourcing. In Proceedings of the 32nd IEEE International Conference on Data Engineering, Vol. , , pp. 49–60. Note: External Links: Document Cited by: §3.1.
  • [15] Y. Tong, Y. Yuan, Y. Cheng, L. Chen, and G. Wang (2017) A survey of spatiotemporal crowdsourced data management techniques. Journal of Software (in Chinese) 28 (1), pp. 35–58. External Links: Document Cited by: §3.1.
  • [16] Y. Tong, X. Zhang, and L. Chen (2016) Tracking frequent items over distributed probabilistic data. World Wide Web Journal 19 (4), pp. 579–604. External Links: Document Cited by: §3.1.
  • [17] Y. Wang, D. Zhang, Q. Liu, F. Shen, and L. H. Lee (2016) Towards enhancing the last-mile delivery: an effective crowd-tasking model with scalable solutions. Transportation Research Part E: Logistics and Transportation Review 93 (), pp. 279–293. External Links: Document Cited by: §1, §1, §3.2.
  • [18] C.J. Zhang, Y. Tong, and L. Chen (2014) Where to: crowd-aided path selection. Proceedings of the VLDB Endowment 7 (11), pp. 2005–2016. External Links: Document Cited by: §3.1.
  • [19] Y. Zheng, L. Zhang, X. Xie, and W. Ma (2009) Mining interesting locations and travel sequences from gps trajectories. In Proceedings of the 18th International Conference on World Wide Web, pp. 791–800. Cited by: §5.1, §5.2.
  • [20] Y. Zheng, J. Wang, G. Li, R. Cheng, and J. Feng (2015) QASCA: a quality-aware task assignment system for crowdsourcing applications. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, pp. 1031–1046. Cited by: §3.1.
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
""
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
   
Add comment
Cancel
Loading ...
398307
This is a comment super asjknd jkasnjk adsnkj
Upvote
Downvote
""
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters
Submit
Cancel

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test
Test description