Sequence labeling, in which a class or label is assigned to each token in a given input order, is a fundamental task in natural language processing. Many advanced neural network architectures have recently been proposed to solve the sequential labeling problem affecting this task. By contrast, only a few approaches have been proposed to address the sequential ensemble problem. In this paper, we resolve the sequential ensemble problem by applying the sequential alignment method in a proposed ensemble framework. Specifically, we propose a simple but efficient ensemble candidate generation framework with which multiple heterogeneous systems can easily be prepared from a single neural sequence labeling network. To evaluate the proposed framework, experiments were conducted with part-of-speech (POS) tagging and dependency label prediction problems. The results indicate that the proposed framework achieved accuracy values that were higher by 0.19 and 0.33 than those achieved by the hard-voting method on the Penn-treebank POS-tagged and Universal dependency-tagged datasets, respectively.