本系列文章針對的資料集是Cityscapes,後續會在Mapillary vista街道資料集上嘗試訓練出一版模型。

1。 資料集和標註檔案

Cityscapes資料夾下

├── annotations 【三個標註檔案】

├── disparity 【額外下載的深度資料集】

├── gtFine 【語義分割和例項分割標註圖片】

├── images 【作者指令碼生成的資料夾,內含5000張影象】

├── labels 【作者指令碼生成的資料夾,內含2975+500張標註影象】

├── leftImg8bit 【原始影象】

└── panoptic 【全景分割標註影象】

1。1 annotations資料夾

├── cityscapes_fine_val。json

├── instancesonly_gtFine_train。json

└── instancesonly_gtFine_val。json

instancesonly_gtFine_train。json是訓練集資料的例項標註, 格式如下:

{

“images”: list, 存放影象的資訊

“categories”: list, 存放類別資訊

“annotations”: list, 存放例項的標註資訊

}

i)image中的資料示例:

{“id”: 500,

“width”: 2048,

“height”: 1024,

“file_name”: “aachen_000000_000019_leftImg8bit。png”,

“seg_file_name”: “aachen_000000_000019_gtFine_instanceIds。png”}

{}

。。。

ii)categories:

{“id”: 1,

“name”: ‘person’},

{“id”: 2,

“name”: ‘rider’},

{},

。。。

iii) annotations

{‘id’: 10118,

‘image_id’: 500,

‘segmentation’: [[903, 446, 902, 447, 901, 448, 900, 449, 900, 450, 899

‘category_id’: 1,

‘iscrowd’: 0,

‘area’: 665,

‘bbox’: [892。0, 446。0, 21。0, 52。0]},

{},

。。。

同一張影象,可有多個標註,對應不同的例項物體。

segmentation的標註方式為polygon,每兩個數字對應一個座標,例如座標(903, 446)是該例項mask邊緣的第一個點。同樣,同一個例項的mask可能由多個polygon組成。

1。2 images

示例:

全景分割 UPSNet 原始碼分析 (1) - 資料格式

1。3 labels

255表示空類,其他非空類的數值是各自的id,由於灰度值很小,所以看起來像全黑的。。。。

示例:

全景分割 UPSNet 原始碼分析 (1) - 資料格式

1。4 panoptic

示例

全景分割 UPSNet 原始碼分析 (1) - 資料格式

2。 模型輸入資料格式

2。1 初始化

首先作者利用coco資料集的處理指令碼,在Cityscapes類中初始化生成了roidb,該list存放了每張訓練影象的標註資訊,具體如下:

{‘id’: 500,

‘width’: 2048,

‘height’: 1024,

‘file_name’: ‘aachen_000000_000019_leftImg8bit。png’,

‘seg_file_name’: ‘aachen_000000_000019_gtFine_instanceIds。png’,

‘dataset’:

‘image’: ‘/home/lin/Documents/dataset/Cityscapes/images/aachen_000000_000019_leftImg8bit。png’,

‘flipped’: False,

‘has_visible_keypoints’: False,

‘boxes’: array([[ 892。, 446。, 912。, 497。],

[ 901。, 444。, 934。, 497。],

[1831。, 440。, 1887。, 547。],

[1966。, 488。, 2042。, 521。],

[1512。, 446。, 1660。, 498。],

[1479。, 456。, 1634。, 513。],

[1876。, 482。, 1921。, 486。],

[1876。, 486。, 1955。, 512。],

[1721。, 450。, 1850。, 508。],

[ 145。, 429。, 304。, 501。],

[ 609。, 420。, 807。, 531。],

[ 792。, 446。, 828。, 489。],

[1839。, 468。, 1877。, 546。]], dtype=float32),

‘segms’: [[[903, 446, 902, 447, 901, 448, 900, 449, 900, 450, 899, 451, 899, 452, 899, 453, 899, 454, 899, 455, 898, 456, 898, 457, 897, 458, 896, 458, 895, 459, 894, 459, 894, 460, 894, 461, 894, 462, 894, 463, 894, 464, 894, 465, 894, 466, 893, 467, 893, 468, 893, 469, 892, 470, 892, 471, 892, 472, 892, 473, 892, 474, 892, 475, 892, 476, 892, 477, 892, 478, 892, 479, 892, 480, 892, 481, 892, 482, 892, 483, 892, 484, 892, 485, 892, 486, 892, 487, 892, 488, 892, 489, 892, 490, 892, 491, 893, 492, 893, 493, 893, 494, 893, 495, 893, 496, 894, 496, 895, 496, 896, 496, 897, 496, 898, 496, 899, 496, 900, 497, 901, 497, 902, 497, 903, 497, 904, 497, 903, 496, 902, 495, 902, 494, 901, 493, 901, 492, 901, 491, 900, 490, 900, 489, 900, 488, 900, 487, 900, 486, 901, 485, 901, 484, 901, 483, 902, 482, 903, 481, 904, 480, 905, 479, 906, 479, 907, 478, 908, 477, 909, 476, 910, 475, 911, 475, 911, 474, 911, 473, 911, 472, 911, 471, 911, 470, 911, 469, 911, 468, 911, 467, 911, 466, 911, 465, 912, 464, 912, 463, 912, 462, 911, 461, 910, 460, 909, 459, 909, 458, 908, 457, 907, 456, 906, 455, 905, 454, 905, 453, 906, 452, 906, 451, 906, 450, 907, 449, 907, 448, 906, 447, 905, 446, 904, 446]], [[910, 476, 909, 477, 908, 478, 907, 479, 906, 480, 905, 480, 904, 481, 903, 482, 902, 483, 902, 484, 902, 485, 901, 486, 901, 487, 901, 488, 901, 489, 901, 490, 902, 491, 902, 492, 902, 493, 903, 494, 903, 495, 904, 496, 905, 496, 906, 497, 907, 497, 908, 497, 909, 497, 910, 497, 911, 497, 911, 496, 911, 495, 911, 494, 911, 493, 911, 492, 911, 491, 911, 490, 911, 489, 911, 488, 911, 487, 911, 486, 911, 485, 911, 484, 911, 483, 911, 482, 911, 481, 911, 480, 911, 479, 911, 478, 911, 477, 911, 476], [922, 444, 921, 445, 920, 445, 919, 445, 918, 446, 918, 447, 918, 448, 918, 449, 919, 450, 919, 451, 919, 452, 919, 453, 919, 454, 919, 455, 918, 456, 918, 457, 918, 458, 918, 459, 917, 460, 917, 461, 916, 462, 916, 463, 915, 464, 916, 464, 917, 465, 917, 466, 917, 467, 917, 468, 917, 469, 917, 470, 917, 471, 917, 472, 917, 473, 917, 474, 917, 475, 917, 476, 917, 477, 917, 478, 917, 479, 917, 480, 917, 481, 917, 482, 917, 483, 917, 484, 917, 485, 917, 486, 917, 487, 917, 488, 917, 489, 917, 490, 917, 491, 917, 492, 917, 493, 917, 494, 917, 495, 917, 496, 918, 496, 919, 496, 920, 497, 921, 497, 922, 497, 923, 497, 924, 497, 925, 497, 926, 497, 927, 497, 928, 497, 929, 497, 930, 497, 931, 497, 931, 496, 930, 495, 930, 494, 930, 493, 929, 492, 929, 491, 929, 490, 929, 489, 929, 488, 928, 487, 928, 486, 928, 485, 928, 484, 928, 483, 928, 482, 928, 481, 927, 480, 927, 479, 927, 478, 927, 477, 928, 476, 929, 475, 930, 475, 931, 474, 931, 473, 932, 472, 932, 471, 932, 470, 932, 469, 932, 468, 932, 467, 932, 466, 932, 465, 932, 464, 932, 463, 933, 462, 933, 461, 933, 460, 934, 459, 934, 458, 934, 457, 934, 456, 934, 455, 934, 454, 934, 453, 934, 452, 933, 452, 932, 452, 931, 451, 930, 451, 929, 451, 928, 450, 928, 449, 927, 448, 927, 447, 926, 446, 925, 445, 924, 444, 923, 444]], [[1871, 440, 1870, 441, 1869, 442, 1869, 443, 1869, 444, 1869, 445, 1869, 446, 1870, 447, 1870, 448, 1870, 449, 1869, 450, 1868, 449, 1867, 449, 1866, 449, 1865, 449, 1864, 449, 1863, 449, 1862, 450, 1861, 450, 1860, 450, 1859, 450, 1858, 451, 1857, 451, 1856, 451, 1855, 452, 1854, 452, 1853, 452, 1852, 452, 1851, 453, 1850, 453, 1849, 453, 1848, 453, 1847, 454, 1846, 454, 1845, 454, 1844, 455, 1843, 456, 1842, 456, 1841, 457, 1840, 458, 1839, 459, 1839, 460, 1838, 461, 1838, 462, 1838, 463, 1838, 464, 1839, 465, 1840, 466, 1841, 467, 1842, 468, 1843, 469, 1844, 469, 1845, 470, 1846, 471, 1847, 472, 1848, 472, 1849, 473, 1850, 474, 1851, 475, 1850, 476, 1849, 477, 1848, 478, 1848, 479, 1847, 480, 1846, 481, 1846, 482, 1845, 483, 1845, 484, 1844, 485, 1843, 486, 1843, 487, 1842, 488, 1842, 489, 1842, 490, 1841, 491, 1841, 492, 1841, 493, 1841, 494, 1840, 495, 1840, 496, 1840, 497, 1840, 498, 1840, 499, 1839, 500, 1839, 501, 1839, 502, 1839, 503, 1838, 504, 1838, 505, 1838, 506, 1838, 507, 1837, 508, 1837, 509, 1837, 510, 1837, 511, 1837, 512, 1836, 513, 1836, 514, 1836, 515, 1835, 516, 1835, 517, 1835, 518, 1834, 519, 1834, 520, 1833, 521, 1833, 522, 1833, 523, 1832, 524, 1832, 525, 1832, 526, 1832, 527, 1832, 528, 1832, 529, 1831, 530, 1831, 531, 1831, 532, 1831, 533, 1831, 534, 1832, 535, 1833, 535, 1834, 535, 1835, 536, 1836, 536, 1837, 536, 1838, 537, 1839, 537, 1840, 537, 1840, 536, 1839, 535, 1839, 534, 1838, 533, 1838, 532, 1838, 531, 1838, 530, 1838, 529, 1838, 528, 1838, 527, 1838, 526, 1838, 525, 1838, 524, 1838, 523, 1838, 522, 1838, 521, 1838, 520, 1839, 519, 1839, 518, 1839, 517, 1839, 516, 1839, 515, 1839, 514, 1839, 513, 1839, 512, 1839, 511, 1839, 510, 1839, 509, 1839, 508, 1839, 507, 1839, 506, 1839, 505, 1840, 504, 1840, 503, 1840, 502, 1840, 501, 1840, 500, 1841, 499, 1841, 498, 1841, 497, 1841, 496, 1841, 495, 1842, 494, 1842, 493, 1842, 492, 1843, 491, 1844, 490, 1845, 489, 1846, 488, 1847, 487, 1848, 486, 1849, 485, 1850, 484, 1851, 483, 1851, 482, 1852, 481, 1852, 480, 1853, 479, 1853, 478, 1852, 477, 1852, 476, 1852, 475, 1851, 474, 1852, 473, 1853, 472, 1854, 472, 1854, 471, 1855, 470, 1856, 469, 1857, 469, 1858, 468, 1859, 468, 1860, 468, 1861, 468, 1862, 468, 1863, 467, 1864, 467, 1865, 467, 1866, 467, 1867, 468, 1867, 469, 1867, 470, 1868, 471, 1868, 472, 1868, 473, 1868, 474, 1869, 474, 1870, 474, 1871, 473, 1872, 473, 1873, 473, 1874, 473, 1875, 473, 1876, 473, 1877, 474, 1878, 475, 1877, 476, 1876, 477, 1875, 477, 1874, 478, 1873, 478, 1872, 478, 1871, 479, 1870, 479, 1869, 480, 1868, 481, 1867, 482, 1866, 483, 1866, 484, 1865, 485, 1865, 486, 1865, 487, 1864, 488, 1864, 489, 1863, 490, 1864, 491, 1865, 491, 1866, 492, 1867, 493, 1867, 494, 1866, 495, 1866, 496, 1866, 497, 1866, 498, 1866, 499, 1866, 500, 1867, 501, 1867, 502, 1867, 503, 1868, 504, 1868, 505, 1868, 506, 1869, 507, 1869, 508, 1869, 509, 1869, 510, 1869, 511, 1869, 512, 1869, 513, 1869, 514, 1869, 515, 1869, 516, 1869, 517, 1869, 518, 1869, 519, 1869, 520, 1869, 521, 1869, 522, 1869, 523, 1869, 524, 1869, 525, 1869, 526, 1869, 527, 1869, 528, 1869, 529, 1869, 530, 1869, 531, 1869, 532, 1869, 533, 1869, 534, 1869, 535, 1868, 536, 1868, 537, 1868, 538, 1867, 539, 1867, 540, 1867, 541, 1866, 542, 1866, 543, 1867, 544, 1868, 544, 1869, 545, 1870, 545, 1871, 546, 1872, 546, 1873, 546, 1874, 546, 1875, 546, 1876, 546, 1877, 547, 1878, 547, 1879, 547, 1880, 547, 1881, 547, 1882, 547, 1881, 546, 1880, 545, 1880, 544, 1879, 543, 1879, 542, 1878, 541, 1877, 540, 1877, 539, 1877, 538, 1877, 537, 1878, 536, 1878, 535, 1878, 534, 1878, 533, 1878, 532, 1878, 531, 1879, 530, 1879, 529, 1879, 528, 1879, 527, 1879, 526, 1879, 525, 1879, 524, 1879, 523, 1879, 522, 1880, 521, 1880, 520, 1880, 519, 1880, 518, 1880, 517, 1880, 516, 1880, 515, 1880, 514, 1880, 513, 1880, 512, 1880, 511, 1880, 510, 1880, 509, 1880, 508, 1880, 507, 1880, 506, 1879, 505, 1879, 504, 1879, 503, 1879, 502, 1879, 501, 1878, 500, 1878, 499, 1878, 498, 1878, 497, 1877, 496, 1877, 495, 1877, 494, 1876, 493, 1876, 492, 1876, 491, 1876, 490, 1876, 489, 1875, 488, 1875, 487, 1875, 486, 1875, 485, 1876, 484, 1877, 484, 1878, 483, 1879, 483, 1880, 483, 1881, 482, 1882, 481, 1883, 480, 1884, 479, 1885, 478, 1885, 477, 1885, 476, 1886, 475, 1886, 474, 1886, 473, 1886, 472, 1886, 471, 1886, 470, 1886, 469, 1886, 468, 1887, 467, 1887, 466, 1887, 465, 1887, 464, 1887, 463, 1887, 462, 1887, 461, 1887, 460, 1887, 459, 1886, 458, 1886, 457, 1886, 456, 1886, 455, 1886, 454, 1885, 453, 1885, 452, 1886, 451, 1886, 450, 1886, 449, 1886, 448, 1885, 448, 1884, 448, 1883, 448, 1882, 447, 1881, 446, 1880, 445, 1880, 444, 1879, 443, 1878, 442, 1877, 442, 1876, 441, 1875, 441, 1874, 441, 1873, 440, 1872, 440]], [[1996, 488, 1995, 489, 1994, 489, 1993, 489, 1992, 489, 1991, 490, 1990, 490, 1989, 490, 1988, 491, 1987, 491, 1986, 491, 1985, 492, 1984, 492, 1983, 492, 1982, 493, 1981, 493, 1980, 494, 1979, 494, 1978, 494, 1977, 495, 1976, 495, 1975, 496, 1974, 496, 1973, 496, 1972, 497, 1971, 497, 1970, 497, 1969, 497, 1968, 497, 1967, 497, 1966, 497, 1966, 498, 1966, 499, 1966, 500, 1966, 501, 1966, 502, 1966, 503, 1966, 504, 1967, 505, 1967, 506, 1967, 507, 1967, 508, 1967, 509, 1967, 510, 1967, 511, 1967, 512, 1967, 513, 1967, 514, 1968, 514, 1969, 514, 1970, 514, 1971, 514, 1972, 514, 1973, 514, 1974, 514, 1975, 515, 1976, 515, 1977, 515, 1978, 515, 1979, 515, 1980, 515, 1981, 515, 1982, 515, 1983, 515, 1984, 516, 1985, 516, 1986, 516, 1987, 516, 1988, 516, 1989, 516, 1990, 516, 1991, 516, 1992, 516, 1993, 517, 1994, 517, 1995, 517, 1996, 517, 1997, 517, 1998, 517, 1999, 517, 2000, 517, 2001, 517, 2002, 518, 2003, 518, 2004, 518, 2005, 518, 2006, 518, 2007, 518, 2008, 518, 2009, 518, 2010, 518, 2011, 519, 2012, 519, 2013, 519, 2014, 519, 2015, 519, 2016, 519, 2017, 519, 2018, 519, 2019, 519, 2020, 519, 2021, 519, 2022, 519, 2023, 520, 2024, 520, 2025, 520, 2026, 520, 2027, 520, 2028, 520, 2029, 520, 2030, 520, 2031, 520, 2032, 520, 2033, 520, 2034, 520, 2035, 520, 2036, 520, 2037, 520, 2038, 520, 2039, 520, 2040, 521, 2041, 521, 2042, 521, 2042, 520, 2042, 519, 2042, 518, 2042, 517, 2042, 516, 2042, 515, 2042, 514, 2042, 513, 2042, 512, 2042, 511, 2042, 510, 2042, 509, 2042, 508, 2042, 507, 2042, 506, 2042, 505, 2042, 504, 2042, 503, 2042, 502, 2042, 501, 2042, 500, 2042, 499, 2042, 498, 2042, 497, 2041, 496, 2041, 495, 2040, 494, 2039, 493, 2038, 492, 2037, 491, 2036, 490, 2035, 490, 2034, 490, 2033, 490, 2032, 490, 2031, 489, 2030, 489, 2029, 489, 2028, 489, 2027, 489, 2026, 489, 2025, 489, 2024, 489, 2023, 489, 2022, 488, 2021, 488, 2020, 488, 2019, 488, 2018, 488, 2017, 488, 2016, 488, 2015, 488, 2014, 488, 2013, 488, 2012, 488, 2011, 488, 2010, 488, 2009, 488, 2008, 488, 2007, 488, 2006, 488, 2005, 488, 2004, 488, 2003, 488, 2002, 488, 2001, 488, 2000, 488, 1999, 488, 1998, 488, 1997, 488]], [[1537, 447, 1536, 448, 1535, 448, 1534, 448, 1533, 448, 1532, 449, 1531, 449, 1530, 449, 1529, 450, 1528, 450, 1527, 450, 1526, 451, 1525, 451, 1524, 451, 1523, 452, 1522, 452, 1521, 453, 1520, 454, 1519, 455, 1518, 456, 1517, 457, 1516, 458, 1515, 458, 1514, 458, 1513, 458, 1513, 459, 1513, 460, 1513, 461, 1513, 462, 1513, 463, 1513, 464, 1513, 465, 1513, 466, 1513, 467, 1512, 468, 1512, 469, 1512, 470, 1512, 471, 1513, 472, 1514, 472, 1515, 473, 1516, 472, 1517, 472, 1518, 471, 1519, 470, 1520, 470, 1521, 469, 1522, 468, 1523, 467, 1524, 467, 1525, 466, 1526, 465, 1527, 465, 1528, 464, 1529, 464, 1530, 464, 1531, 463, 1532, 463, 1533, 463, 1534, 462, 1535, 462, 1536, 461, 1537, 461, 1537, 460, 1537, 459, 1537, 458, 1537, 457, 1537, 456, 1537, 455, 1537, 454, 1537, 453, 1537, 452, 1537, 451, 1537, 450, 1537, 449, 1537, 448], [1545, 446, 1545, 447, 1545, 448, 1545, 449, 1545, 450, 1545, 451, 1545, 452, 1545, 453, 1545, 454, 1545, 455, 1545, 456, 1545, 457, 1545, 458, 1546, 458, 1547, 458, 1548, 458, 1549, 458, 1550, 457, 1551, 457, 1552, 457, 1553, 457, 1554, 457, 1555, 457, 1556, 457, 1557, 456, 1558, 456, 1559, 456, 1560, 456, 1561, 456, 1562, 456, 1563, 456, 1564, 456, 1565, 456, 1566, 455, 1567, 455, 1568, 455, 1569, 455, 1570, 455, 1571, 455, 1572, 455, 1573, 455, 1574, 455, 1575, 455, 1576, 455, 1577, 455, 1578, 455, 1579, 455, 1580, 455, 1581, 455, 1582, 455, 1583, 455, 1584, 455, 1585, 455, 1586, 455, 1587, 455, 1588, 455, 1589, 455, 1590, 455, 1591, 455, 1592, 455, 1593, 455, 1594, 455, 1595, 456, 1596, 456, 1597, 456, 1598, 456, 1599, 456, 1600, 456, 1601, 456, 1602, 456, 1603, 456, 1604, 456, 1605, 456, 1606, 457, 1607, 457, 1608, 457, 1609, 457, 1610, 457, 1611, 457, 1612, 457, 1613, 458, 1614, 458, 1615, 458, 1616, 459, 1617, 459, 1618, 459, 1619, 460, 1620, 460, 1621, 461, 1622, 462, 1623, 463, 1624, 464, 1624, 465, 1625, 466, 1626, 467, 1626, 468, 1627, 469, 1627, 470, 1627, 471, 1628, 472, 1628, 473, 1628, 474, 1629, 475, 1629, 476, 1629, 477, 1630, 478, 1630, 479, 1631, 480, 1631, 481, 1631, 482, 1632, 483, 1632, 484, 1633, 485, 1633, 486, 1633, 487, 1634, 488, 1634, 489, 1634, 490, 1635, 491, 1635, 492, 1635, 493, 1635, 494, 1635, 495, 1635, 496, 1636, 496, 1637, 496, 1638, 497, 1639, 497, 1640, 497, 1641, 498, 1642, 498, 1643, 498, 1644, 498, 1645, 497, 1646, 497, 1647, 497, 1648, 496, 1649, 496, 1649, 495, 1649, 494, 1650, 493, 1650, 492, 1650, 491, 1651, 490, 1652, 490, 1653, 490, 1654, 490, 1655, 490, 1656, 490, 1657, 490, 1658, 490, 1659, 490, 1659, 489, 1659, 488, 1659, 487, 1659, 486, 1659, 485, 1660, 484, 1660, 483, 1660, 482, 1660, 481, 1660, 480, 1660, 479, 1659, 478, 1658, 478, 1657, 477, 1656, 476, 1655, 476, 1654, 475, 1653, 475, 1652, 474, 1651, 473, 1650, 473, 1649, 472, 1648, 472, 1647, 472, 1646, 471, 1645, 471, 1644, 471, 1643, 471, 1642, 471, 1641, 470, 1640, 470, 1639, 470, 1638, 470, 1637, 470, 1636, 470, 1635, 470, 1634, 470, 1633, 471, 1632, 471, 1631, 471, 1630, 470, 1629, 469, 1628, 468, 1627, 467, 1626, 466, 1625, 465, 1624, 464, 1623, 463, 1622, 462, 1621, 461, 1620, 460, 1619, 459, 1618, 458, 1617, 457, 1616, 457, 1615, 457, 1614, 456, 1613, 456, 1612, 456, 1611, 455, 1610, 455, 1609, 455, 1608, 454, 1607, 454, 1606, 454, 1605, 453, 1604, 453, 1603, 453, 1602, 452, 1601, 452, 1600, 452, 1599, 451, 1598, 451, 1597, 451, 1596, 451, 1595, 451, 1594, 450, 1593, 450, 1592, 450, 1591, 450, 1590, 450, 1589, 449, 1588, 449, 1587, 449, 1586, 449, 1585, 449, 1584, 448, 1583, 448, 1582, 448, 1581, 448, 1580, 448, 1579, 447, 1578, 447, 1577, 447, 1576, 447, 1575, 447, 1574, 447, 1573, 447, 1572, 447, 1571, 447, 1570, 447, 1569, 447, 1568, 447, 1567, 447, 1566, 447, 1565, 447, 1564, 447, 1563, 447, 1562, 447, 1561, 447, 1560, 447, 1559, 447, 1558, 447, 1557, 447, 1556, 447, 1555, 447, 1554, 447, 1553, 447, 1552, 447, 1551, 446, 1550, 446, 1549, 446, 1548, 446, 1547, 446, 1546, 446]], [[1536, 462, 1535, 463, 1534, 463, 1533, 464, 1532, 464, 1531, 464, 1530, 465, 1529, 465, 1528, 465, 1527, 466, 1526, 466, 1525, 467, 1524, 468, 1523, 468, 1522, 469, 1521, 470, 1520, 471, 1519, 471, 1518, 472, 1517, 473, 1516, 473, 1515, 474, 1514, 475, 1513, 476, 1512, 476, 1511, 476, 1510, 476, 1509, 476, 1508, 476, 1507, 476, 1506, 476, 1505, 476, 1504, 476, 1503, 476, 1502, 476, 1501, 476, 1500, 476, 1499, 476, 1498, 476, 1497, 477, 1496, 477, 1495, 477, 1494, 477, 1493, 477, 1492, 478, 1491, 478, 1490, 478, 1489, 478, 1488, 478, 1487, 479, 1486, 479, 1485, 480, 1484, 481, 1483, 482, 1482, 483, 1482, 484, 1481, 485, 1480, 486, 1479, 487, 1479, 488, 1479, 489, 1480, 490, 1480, 491, 1480, 492, 1480, 493, 1480, 494, 1481, 495, 1481, 496, 1481, 497, 1481, 498, 1482, 499, 1482, 500, 1483, 500, 1484, 500, 1485, 500, 1486, 500, 1487, 501, 1488, 502, 1489, 502, 1490, 503, 1491, 503, 1492, 503, 1493, 504, 1494, 504, 1495, 505, 1496, 505, 1497, 505, 1498, 506, 1499, 506, 1500, 506, 1501, 506, 1502, 506, 1503, 505, 1504, 505, 1505, 505, 1506, 505, 1507, 505, 1508, 505, 1509, 504, 1510, 504, 1511, 504, 1512, 504, 1513, 504, 1514, 504, 1515, 504, 1516, 504, 1517, 504, 1518, 504, 1519, 504, 1520, 504, 1521, 504, 1522, 504, 1523, 504, 1524, 505, 1524, 506, 1525, 507, 1525, 508, 1526, 509, 1527, 509, 1528, 510, 1529, 511, 1530, 511, 1531, 512, 1532, 512, 1533, 512, 1533, 511, 1533, 510, 1533, 509, 1534, 508, 1534, 507, 1534, 506, 1534, 505, 1534, 504, 1534, 503, 1534, 502, 1534, 501, 1534, 500, 1535, 499, 1535, 498, 1535, 497, 1535, 496, 1535, 495, 1535, 494, 1535, 493, 1535, 492, 1535, 491, 1536, 490, 1536, 489, 1536, 488, 1536, 487, 1536, 486, 1536, 485, 1537, 484, 1537, 483, 1537, 482, 1538, 481, 1538, 480, 1538, 479, 1538, 478, 1538, 477, 1538, 476, 1538, 475, 1538, 474, 1538, 473, 1538, 472, 1538, 471, 1538, 470, 1538, 469, 1538, 468, 1538, 467, 1538, 466, 1538, 465, 1537, 464, 1537, 463, 1537, 462], [1566, 456, 1565, 457, 1564, 457, 1563, 457, 1562, 457, 1561, 457, 1560, 457, 1559, 457, 1558, 457, 1557, 457, 1556, 458, 1555, 458, 1554, 458, 1553, 458, 1552, 458, 1551, 458, 1550, 458, 1549, 459, 1548, 459, 1547, 459, 1546, 459, 1545, 459, 1545, 460, 1545, 461, 1545, 462, 1545, 463, 1545, 464, 1545, 465, 1545, 466, 1545, 467, 1545, 468, 1545, 469, 1545, 470, 1545, 471, 1545, 472, 1545, 473, 1545, 474, 1545, 475, 1545, 476, 1545, 477, 1545, 478, 1545, 479, 1545, 480, 1545, 481, 1546, 482, 1546, 483, 1547, 484, 1547, 485, 1547, 486, 1547, 487, 1547, 488, 1547, 489, 1548, 490, 1548, 491, 1548, 492, 1548, 493, 1548, 494, 1548, 495, 1548, 496, 1548, 497, 1548, 498, 1548, 499, 1549, 500, 1549, 501, 1549, 502, 1549, 503, 1549, 504, 1550, 503, 1551, 503, 1552, 503, 1553, 503, 1554, 503, 1555, 503, 1556, 504, 1557, 504, 1558, 504, 1559, 504, 1560, 504, 1561, 504, 1562, 504, 1563, 504, 1564, 505, 1565, 505, 1566, 505, 1567, 505, 1568, 505, 1569, 505, 1570, 505, 1571, 505, 1572, 506, 1573, 506, 1574, 506, 1575, 506, 1576, 506, 1577, 506, 1578, 506, 1579, 506, 1580, 506, 1581, 506, 1582, 506, 1583, 506, 1584, 506, 1585, 506, 1586, 506, 1587, 506, 1588, 506, 1589, 506, 1590, 506, 1591, 506, 1592, 506, 1593, 506, 1594, 506, 1595, 506, 1596, 506, 1597, 506, 1598, 506, 1599, 506, 1600, 506, 1601, 506, 1602, 506, 1603, 506, 1604, 506, 1605, 506, 1606, 506, 1607, 506, 1608, 506, 1609, 507, 1610, 508, 1611, 509, 1612, 510, 1613, 511, 1614, 512, 1615, 513, 1616, 513, 1617, 513, 1618, 513, 1619, 513, 1620, 513, 1621, 513, 1622, 513, 1623, 513, 1624, 513, 1625, 513, 1626, 512, 1627, 511, 1627, 510, 1628, 509, 1629, 508, 1630, 507, 1631, 506, 1632, 505, 1633, 504, 1634, 503, 1634, 502, 1634, 501, 1634, 500, 1634, 499, 1634, 498, 1634, 497, 1634, 496, 1634, 495, 1634, 494, 1634, 493, 1634, 492, 1634, 491, 1633, 490, 1633, 489, 1633, 488, 1632, 487, 1632, 486, 1632, 485, 1631, 484, 1631, 483, 1630, 482, 1630, 481, 1630, 480, 1629, 479, 1629, 478, 1628, 477, 1628, 476, 1628, 475, 1627, 474, 1627, 473, 1627, 472, 1626, 471, 1626, 470, 1626, 469, 1625, 468, 1625, 467, 1624, 466, 1623, 465, 1623, 464, 1622, 463, 1621, 462, 1620, 461, 1619, 461, 1618, 460, 1617, 460, 1616, 460, 1615, 459, 1614, 459, 1613, 459, 1612, 458, 1611, 458, 1610, 458, 1609, 458, 1608, 458, 1607, 458, 1606, 458, 1605, 457, 1604, 457, 1603, 457, 1602, 457, 1601, 457, 1600, 457, 1599, 457, 1598, 457, 1597, 457, 1596, 457, 1595, 457, 1594, 456, 1593, 456, 1592, 456, 1591, 456, 1590, 456, 1589, 456, 1588, 456, 1587, 456, 1586, 456, 1585, 456, 1584, 456, 1583, 456, 1582, 456, 1581, 456, 1580, 456, 1579, 456, 1578, 456, 1577, 456, 1576, 456, 1575, 456, 1574, 456, 1573, 456, 1572, 456, 1571, 456, 1570, 456, 1569, 456, 1568, 456, 1567, 456]], [[1900, 483, 1900, 484, 1900, 485, 1901, 485, 1902, 485, 1903, 485, 1904, 485, 1905, 485, 1906, 485, 1907, 485, 1908, 485, 1909, 485, 1910, 485, 1911, 485, 1912, 485, 1913, 485, 1914, 485, 1915, 485, 1916, 485, 1917, 485, 1918, 485, 1919, 485, 1920, 485, 1921, 485, 1920, 485, 1919, 485, 1918, 484, 1917, 484, 1916, 484, 1915, 484, 1914, 484, 1913, 484, 1912, 484, 1911, 484, 1910, 484, 1909, 483, 1908, 483, 1907, 483, 1906, 483, 1905, 483, 1904, 483, 1903, 483, 1902, 483, 1901, 483], [1882, 482, 1881, 483, 1880, 484, 1879, 484, 1878, 484, 1877, 485, 1876, 485, 1876, 486, 1877, 486, 1878, 486, 1879, 486, 1880, 486, 1881, 486, 1882, 486, 1883, 486, 1884, 486, 1885, 486, 1886, 486, 1887, 486, 1888, 486, 1889, 486, 1890, 486, 1891, 486, 1891, 485, 1891, 484, 1891, 483, 1891, 482, 1890, 482, 1889, 482, 1888, 482, 1887, 482, 1886, 482, 1885, 482, 1884, 482, 1883, 482]], [[1934, 490, 1934, 491, 1934, 492, 1934, 493, 1934, 494, 1934, 495, 1934, 496, 1934, 497, 1934, 498, 1934, 499, 1934, 500, 1934, 501, 1934, 502, 1934, 503, 1934, 504, 1934, 505, 1934, 506, 1934, 507, 1934, 508, 1934, 509, 1934, 510, 1934, 511, 1934, 512, 1935, 512, 1936, 512, 1937, 512, 1938, 512, 1939, 512, 1940, 512, 1941, 512, 1942, 512, 1943, 512, 1944, 512, 1945, 512, 1946, 512, 1947, 512, 1948, 512, 1949, 512, 1950, 512, 1951, 512, 1952, 512, 1953, 512, 1954, 512, 1955, 512, 1955, 511, 1955, 510, 1955, 509, 1955, 508, 1955, 507, 1954, 506, 1954, 505, 1954, 504, 1954, 503, 1954, 502, 1954, 501, 1954, 500, 1954, 499, 1953, 499, 1952, 499, 1951, 499, 1950, 498, 1949, 497, 1948, 497, 1947, 496, 1946, 496, 1945, 495, 1944, 494, 1943, 494, 1942, 493, 1941, 493, 1940, 492, 1939, 492, 1938, 492, 1937, 491, 1936, 491, 1935, 490], [1876, 487, 1876, 488, 1877, 489, 1877, 490, 1877, 491, 1877, 492, 1877, 493, 1878, 494, 1878, 495, 1878, 496, 1879, 497, 1879, 498, 1879, 499, 1879, 500, 1880, 501, 1880, 502, 1880, 503, 1880, 504, 1881, 504, 1882, 505, 1883, 505, 1884, 505, 1885, 505, 1886, 505, 1887, 505, 1888, 506, 1889, 506, 1890, 506, 1891, 506, 1892, 506, 1892, 505, 1892, 504, 1892, 503, 1892, 502, 1892, 501, 1892, 500, 1892, 499, 1892, 498, 1892, 497, 1892, 496, 1892, 495, 1892, 494, 1891, 493, 1891, 492, 1891, 491, 1891, 490, 1891, 489, 1891, 488, 1891, 487, 1890, 487, 1889, 487, 1888, 487, 1887, 487, 1886, 487, 1885, 487, 1884, 487, 1883, 487, 1882, 487, 1881, 487, 1880, 487, 1879, 487, 1878, 487, 1877, 487], [1900, 486, 1900, 487, 1900, 488, 1900, 489, 1900, 490, 1900, 491, 1900, 492, 1901, 493, 1901, 494, 1901, 495, 1901, 496, 1901, 497, 1901, 498, 1901, 499, 1901, 500, 1901, 501, 1901, 502, 1901, 503, 1901, 504, 1901, 505, 1901, 506, 1901, 507, 1901, 508, 1902, 508, 1903, 508, 1904, 508, 1905, 508, 1906, 508, 1907, 508, 1908, 509, 1909, 509, 1910, 509, 1911, 509, 1912, 509, 1913, 510, 1914, 510, 1915, 510, 1916, 510, 1917, 511, 1918, 511, 1919, 511, 1920, 511, 1921, 511, 1922, 512, 1923, 512, 1924, 512, 1925, 512, 1925, 511, 1925, 510, 1925, 509, 1925, 508, 1925, 507, 1925, 506, 1925, 505, 1925, 504, 1925, 503, 1925, 502, 1925, 501, 1925, 500, 1925, 499, 1925, 498, 1925, 497, 1925, 496, 1925, 495, 1925, 494, 1925, 493, 1925, 492, 1925, 491, 1925, 490, 1925, 489, 1925, 488, 1925, 487, 1924, 486, 1923, 486, 1922, 486, 1921, 486, 1920, 486, 1919, 486, 1918, 486, 1917, 486, 1916, 486, 1915, 486, 1914, 486, 1913, 486, 1912, 486, 1911, 486, 1910, 486, 1909, 486, 1908, 486, 1907, 486, 1906, 486, 1905, 486, 1904, 486, 1903, 486, 1902, 486, 1901, 486]], [[1772, 450, 1771, 451, 1770, 452, 1769, 453, 1768, 454, 1767, 455, 1766, 456, 1765, 457, 1764, 457, 1763, 458, 1762, 458, 1761, 459, 1760, 459, 1759, 460, 1758, 460, 1757, 461, 1756, 461, 1755, 461, 1754, 462, 1753, 462, 1752, 463, 1751, 463, 1750, 464, 1749, 464, 1748, 465, 1747, 465, 1746, 466, 1745, 466, 1744, 467, 1743, 467, 1742, 468, 1741, 468, 1740, 469, 1739, 469, 1738, 470, 1737, 470, 1736, 471, 1735, 471, 1734, 472, 1733, 472, 1732, 473, 1731, 473, 1730, 474, 1729, 474, 1728, 475, 1727, 475, 1726, 476, 1725, 477, 1725, 478, 1724, 479, 1724, 480, 1724, 481, 1723, 482, 1723, 483, 1723, 484, 1722, 485, 1722, 486, 1722, 487, 1722, 488, 1722, 489, 1721, 490, 1721, 491, 1721, 492, 1722, 493, 1723, 494, 1724, 495, 1725, 495, 1726, 496, 1727, 497, 1728, 497, 1729, 497, 1730, 497, 1731, 498, 1732, 498, 1733, 498, 1734, 498, 1735, 498, 1736, 498, 1737, 498, 1738, 498, 1739, 499, 1740, 499, 1741, 499, 1742, 499, 1743, 499, 1744, 499, 1745, 499, 1746, 499, 1747, 499, 1748, 499, 1749, 500, 1750, 500, 1751, 500, 1752, 500, 1753, 500, 1754, 500, 1755, 500, 1756, 501, 1757, 502, 1757, 503, 1757, 504, 1758, 505, 1758, 506, 1759, 507, 1760, 507, 1761, 507, 1762, 508, 1763, 508, 1764, 508, 1765, 508, 1766, 508, 1767, 508, 1768, 508, 1769, 508, 1770, 508, 1771, 507, 1772, 507, 1773, 507, 1774, 507, 1775, 506, 1776, 505, 1777, 505, 1778, 504, 1779, 503, 1779, 502, 1779, 501, 1779, 500, 1779, 499, 1779, 498, 1780, 497, 1781, 497, 1782, 497, 1783, 497, 1784, 497, 1785, 497, 1786, 497, 1787, 497, 1788, 497, 1789, 497, 1790, 497, 1791, 497, 1792, 497, 1793, 497, 1794, 497, 1795, 497, 1796, 497, 1797, 497, 1798, 497, 1799, 497, 1800, 498, 1801, 498, 1802, 498, 1803, 498, 1804, 498, 1805, 498, 1806, 498, 1807, 498, 1808, 498, 1809, 498, 1810, 498, 1811, 498, 1812, 498, 1813, 498, 1814, 498, 1815, 498, 1816, 498, 1817, 498, 1818, 498, 1819, 498, 1820, 498, 1821, 498, 1822, 498, 1823, 498, 1824, 498, 1825, 498, 1826, 499, 1827, 499, 1828, 499, 1829, 499, 1830, 499, 1831, 499, 1832, 500, 1832, 501, 1832, 502, 1832, 503, 1833, 504, 1833, 505, 1833, 506, 1834, 506, 1835, 506, 1836, 506, 1837, 506, 1837, 505, 1837, 504, 1838, 503, 1838, 502, 1838, 501, 1838, 500, 1839, 499, 1839, 498, 1839, 497, 1839, 496, 1839, 495, 1840, 494, 1840, 493, 1840, 492, 1840, 491, 1841, 490, 1841, 489, 1841, 488, 1842, 487, 1842, 486, 1843, 485, 1844, 484, 1844, 483, 1845, 482, 1845, 481, 1846, 480, 1847, 479, 1847, 478, 1848, 477, 1849, 476, 1850, 475, 1849, 474, 1848, 473, 1847, 473, 1846, 472, 1845, 471, 1844, 470, 1843, 470, 1842, 469, 1841, 468, 1840, 467, 1839, 466, 1838, 465, 1837, 464, 1837, 463, 1837, 462, 1837, 461, 1838, 460, 1838, 459, 1839, 458, 1840, 457, 1841, 456, 1840, 456, 1839, 456, 1838, 455, 1837, 455, 1836, 455, 1835, 455, 1834, 455, 1833, 455, 1832, 455, 1831, 454, 1830, 454, 1829, 454, 1828, 454, 1827, 454, 1826, 454, 1825, 454, 1824, 454, 1823, 453, 1822, 453, 1821, 453, 1820, 453, 1819, 453, 1818, 453, 1817, 453, 1816, 453, 1815, 452, 1814, 452, 1813, 452, 1812, 452, 1811, 452, 1810, 452, 1809, 452, 1808, 452, 1807, 451, 1806, 451, 1805, 451, 1804, 451, 1803, 451, 1802, 451, 1801, 451, 1800, 451, 1799, 450, 1798, 450, 1797, 450, 1796, 450, 1795, 450, 1794, 450, 1793, 450, 1792, 450, 1791, 450, 1790, 450, 1789, 450, 1788, 450, 1787, 450, 1786, 450, 1785, 450, 1784, 450, 1783, 450, 1782, 450, 1781, 450, 1780, 450, 1779, 450, 1778, 450, 1777, 450, 1776, 450, 1775, 450, 1774, 450, 1773, 450]], [[195, 429, 194, 430, 193, 430, 192, 431, 191, 431, 190, 432, 189, 432, 188, 432, 187, 433, 186, 433, 185, 434, 184, 434, 183, 435, 182, 435, 181, 436, 180, 436, 179, 437, 178, 438, 177, 439, 176, 440, 175, 441, 174, 442, 173, 443, 172, 444, 173, 445, 173, 446, 174, 447, 175, 448, 176, 449, 177, 450, 177, 451, 178, 452, 179, 453, 180, 454, 181, 455, 182, 456, 182, 457, 183, 458, 184, 459, 185, 460, 186, 461, 186, 462, 187, 463, 188, 464, 188, 465, 189, 466, 189, 467, 190, 468, 191, 469, 191, 470, 192, 471, 191, 472, 190, 473, 189, 474, 188, 475, 187, 476, 186, 476, 185, 476, 184, 476, 183, 476, 182, 476, 181, 476, 180, 476, 179, 476, 178, 476, 177, 476, 176, 476, 175, 476, 174, 477, 173, 477, 172, 477, 171, 477, 170, 477, 169, 477, 168, 477, 167, 477, 166, 477, 165, 477, 164, 477, 163, 477, 162, 477, 161, 477, 160, 477, 159, 477, 158, 477, 157, 477, 156, 477, 155, 477, 154, 477, 153, 477, 152, 477, 151, 477, 150, 478, 149, 478, 148, 478, 147, 478, 146, 478, 145, 478, 145, 479, 145, 480, 145, 481, 145, 482, 145, 483, 145, 484, 145, 485, 145, 486, 145, 487, 145, 488, 145, 489, 145, 490, 145, 491, 145, 492, 145, 493, 146, 494, 146, 495, 146, 496, 147, 497, 147, 498, 148, 498, 149, 498, 150, 498, 151, 498, 152, 498, 153, 499, 154, 499, 155, 499, 156, 499, 157, 499, 158, 499, 159, 499, 160, 499, 161, 499, 162, 498, 163, 498, 164, 497, 164, 496, 165, 495, 166, 495, 167, 495, 168, 495, 169, 495, 170, 495, 171, 495, 172, 495, 173, 495, 174, 495, 175, 495, 176, 495, 177, 495, 178, 495, 179, 495, 180, 495, 181, 495, 182, 495, 183, 495, 184, 495, 185, 495, 186, 495, 187, 495, 188, 495, 189, 495, 190, 495, 191, 495, 192, 495, 193, 495, 194, 495, 195, 495, 196, 495, 197, 495, 198, 495, 199, 495, 200, 495, 201, 495, 202, 495, 203, 495, 204, 495, 205, 495, 206, 495, 207, 495, 208, 495, 209, 496, 210, 497, 211, 498, 212, 499, 213, 500, 214, 501, 215, 501, 216, 501, 217, 501, 218, 501, 219, 501, 220, 501, 221, 501, 222, 501, 223, 501, 224, 500, 225, 500, 226, 500, 227, 500, 228, 500, 229, 500, 230, 500, 231, 499, 232, 498, 232, 497, 233, 496, 234, 495, 235, 494, 236, 494, 237, 494, 238, 494, 239, 494, 240, 494, 241, 494, 242, 494, 243, 494, 244, 494, 245, 494, 246, 494, 247, 494, 248, 494, 249, 494, 250, 494, 251, 494, 252, 494, 253, 494, 254, 494, 255, 494, 256, 494, 257, 494, 258, 494, 259, 494, 260, 494, 261, 494, 262, 494, 263, 494, 264, 494, 265, 494, 266, 494, 267, 494, 268, 494, 269, 494, 270, 494, 271, 494, 272, 494, 273, 494, 274, 494, 275, 494, 276, 494, 277, 494, 278, 494, 279, 494, 280, 494, 281, 495, 281, 496, 281, 497, 281, 498, 282, 499, 283, 500, 284, 500, 285, 500, 286, 501, 287, 501, 288, 501, 289, 501, 290, 501, 291, 501, 292, 501, 293, 501, 294, 501, 295, 500, 296, 500, 297, 500, 298, 499, 299, 498, 300, 497, 301, 496, 302, 495, 302, 494, 302, 493, 303, 492, 303, 491, 303, 490, 303, 489, 303, 488, 304, 487, 304, 486, 304, 485, 304, 484, 304, 483, 304, 482, 304, 481, 304, 480, 304, 479, 304, 478, 304, 477, 304, 476, 304, 475, 304, 474, 304, 473, 304, 472, 304, 471, 304, 470, 304, 469, 304, 468, 304, 467, 304, 466, 304, 465, 304, 464, 304, 463, 303, 462, 303, 461, 303, 460, 302, 459, 302, 458, 302, 457, 301, 456, 301, 455, 301, 454, 300, 453, 300, 452, 299, 451, 298, 450, 298, 449, 297, 448, 296, 447, 296, 446, 295, 445, 294, 444, 293, 443, 292, 442, 291, 441, 290, 440, 289, 439, 289, 438, 288, 437, 287, 436, 286, 435, 285, 434, 284, 433, 283, 432, 282, 431, 281, 430, 280, 430, 279, 430, 278, 430, 277, 430, 276, 430, 275, 430, 274, 430, 273, 430, 272, 430, 271, 430, 270, 430, 269, 430, 268, 430, 267, 430, 266, 430, 265, 430, 264, 430, 263, 430, 262, 430, 261, 430, 260, 430, 259, 430, 258, 430, 257, 430, 256, 430, 255, 430, 254, 430, 253, 430, 252, 430, 251, 430, 250, 430, 249, 430, 248, 430, 247, 430, 246, 430, 245, 430, 244, 430, 243, 430, 242, 430, 241, 430, 240, 430, 239, 429, 238, 429, 237, 429, 236, 429, 235, 429, 234, 429, 233, 429, 232, 429, 231, 429, 230, 429, 229, 429, 228, 429, 227, 429, 226, 429, 225, 429, 224, 429, 223, 429, 222, 429, 221, 429, 220, 429, 219, 429, 218, 429, 217, 429, 216, 429, 215, 429, 214, 429, 213, 429, 212, 429, 211, 429, 210, 429, 209, 429, 208, 429, 207, 429, 206, 429, 205, 429, 204, 429, 203, 429, 202, 429, 201, 429, 200, 429, 199, 429, 198, 429, 197, 429, 196, 429]], [[673, 420, 672, 421, 671, 422, 670, 423, 669, 423, 668, 424, 667, 424, 666, 425, 665, 425, 664, 426, 663, 426, 662, 427, 661, 427, 660, 428, 659, 428, 658, 429, 657, 430, 656, 431, 655, 432, 655, 433, 654, 434, 653, 435, 653, 436, 652, 437, 651, 438, 650, 439, 650, 440, 649, 441, 648, 442, 648, 443, 647, 444, 646, 445, 645, 446, 645, 447, 644, 448, 643, 449, 643, 450, 643, 451, 644, 452, 644, 453, 644, 454, 644, 455, 643, 456, 642, 455, 641, 455, 640, 455, 639, 454, 638, 453, 637, 452, 636, 451, 635, 450, 634, 450, 633, 450, 632, 450, 631, 450, 630, 450, 629, 450, 628, 450, 627, 451, 626, 452, 625, 453, 626, 454, 627, 455, 627, 456, 628, 457, 629, 458, 628, 459, 627, 460, 626, 460, 625, 461, 624, 461, 623, 462, 622, 463, 621, 463, 620, 464, 619, 465, 619, 466, 618, 467, 617, 468, 617, 469, 616, 470, 616, 471, 615, 472, 614, 473, 614, 474, 613, 475, 613, 476, 612, 477, 612, 478, 611, 479, 610, 480, 610, 481, 609, 482, 609, 483, 609, 484, 609, 485, 609, 486, 609, 487, 609, 488, 609, 489, 609, 490, 609, 491, 609, 492, 610, 493, 610, 494, 610, 495, 610, 496, 610, 497, 610, 498, 610, 499, 610, 500, 610, 501, 610, 502, 610, 503, 610, 504, 610, 505, 610, 506, 610, 507, 610, 508, 610, 509, 610, 510, 610, 511, 611, 512, 611, 513, 611, 514, 611, 515, 611, 516, 611, 517, 611, 518, 611, 519, 611, 520, 611, 521, 612, 522, 613, 523, 614, 524, 615, 525, 616, 525, 617, 525, 618, 525, 619, 525, 620, 525, 621, 525, 622, 525, 623, 525, 624, 525, 625, 524, 626, 524, 627, 524, 628, 524, 629, 524, 630, 524, 631, 524, 632, 524, 632, 523, 633, 522, 633, 521, 633, 520, 634, 519, 634, 518, 634, 517, 634, 516, 635, 515, 635, 514, 636, 513, 637, 513, 638, 513, 639, 513, 640, 513, 641, 514, 642, 514, 643, 514, 644, 514, 645, 514, 646, 514, 647, 514, 648, 514, 649, 514, 650, 514, 651, 514, 652, 514, 653, 515, 654, 515, 655, 515, 656, 515, 657, 515, 658, 515, 659, 515, 660, 515, 661, 515, 662, 515, 663, 515, 664, 515, 665, 516, 666, 516, 667, 516, 668, 516, 669, 516, 670, 516, 671, 517, 671, 518, 672, 519, 672, 520, 672, 521, 673, 522, 673, 523, 673, 524, 674, 525, 674, 526, 675, 527, 676, 527, 677, 528, 678, 529, 679, 530, 680, 531, 681, 531, 682, 531, 683, 531, 684, 531, 685, 531, 686, 531, 687, 531, 688, 531, 689, 531, 690, 531, 691, 531, 692, 530, 693, 530, 694, 530, 695, 529, 696, 529, 697, 528, 698, 527, 698, 526, 699, 525, 700, 524, 700, 523, 701, 522, 701, 521, 702, 520, 703, 519, 704, 518, 705, 517, 706, 516, 707, 516, 708, 516, 709, 517, 710, 518, 711, 519, 711, 520, 712, 521, 713, 522, 714, 523, 715, 523, 716, 523, 717, 523, 718, 523, 719, 523, 720, 522, 721, 522, 722, 522, 723, 522, 724, 522, 725, 522, 726, 522, 727, 522, 728, 522, 729, 521, 729, 520, 730, 519, 730, 518, 731, 517, 731, 516, 732, 515, 733, 514, 734, 514, 735, 514, 736, 514, 737, 514, 738, 514, 739, 514, 740, 514, 741, 514, 742, 514, 743, 514, 744, 514, 745, 514, 746, 514, 747, 514, 748, 514, 749, 514, 750, 514, 751, 514, 752, 514, 753, 514, 754, 514, 755, 514, 756, 514, 757, 514, 758, 514, 759, 514, 760, 514, 761, 514, 762, 514, 763, 514, 764, 514, 765, 514, 766, 514, 767, 514, 768, 515, 769, 516, 769, 517, 770, 518, 771, 519, 772, 519, 773, 519, 774, 519, 775, 519, 776, 520, 776, 521, 777, 522, 777, 523, 777, 524, 778, 525, 778, 526, 779, 526, 780, 526, 781, 527, 782, 527, 783, 527, 784, 527, 785, 527, 786, 527, 787, 527, 788, 527, 789, 527, 790, 527, 791, 527, 792, 527, 793, 527, 794, 527, 795, 527, 796, 527, 797, 527, 798, 526, 799, 526, 800, 525, 801, 524, 802, 524, 803, 523, 803, 522, 803, 521, 804, 520, 804, 519, 804, 518, 804, 517, 804, 516, 804, 515, 804, 514, 805, 513, 805, 512, 805, 511, 805, 510, 805, 509, 805, 508, 805, 507, 806, 506, 806, 505, 806, 504, 806, 503, 806, 502, 806, 501, 806, 500, 806, 499, 806, 498, 806, 497, 806, 496, 806, 495, 806, 494, 806, 493, 806, 492, 807, 491, 807, 490, 807, 489, 807, 488, 807, 487, 807, 486, 807, 485, 807, 484, 807, 483, 807, 482, 807, 481, 807, 480, 807, 479, 807, 478, 806, 477, 806, 476, 806, 475, 805, 474, 805, 473, 805, 472, 804, 471, 804, 470, 804, 469, 804, 468, 803, 467, 803, 466, 803, 465, 802, 464, 802, 463, 802, 462, 802, 461, 801, 460, 801, 459, 801, 458, 800, 457, 800, 456, 799, 455, 798, 455, 797, 454, 796, 454, 795, 454, 794, 453, 793, 452, 792, 451, 791, 450, 791, 449, 790, 448, 789, 447, 789, 446, 788, 445, 788, 444, 787, 443, 787, 442, 786, 441, 786, 440, 785, 439, 785, 438, 784, 437, 783, 436, 783, 435, 782, 434, 782, 433, 781, 432, 781, 431, 780, 430, 779, 429, 778, 428, 777, 427, 776, 427, 775, 426, 774, 425, 773, 425, 772, 424, 771, 423, 770, 422, 769, 422, 768, 421, 767, 420, 766, 420, 765, 420, 764, 420, 763, 420, 762, 420, 761, 420, 760, 420, 759, 420, 758, 420, 757, 420, 756, 420, 755, 420, 754, 420, 753, 420, 752, 420, 751, 420, 750, 420, 749, 420, 748, 420, 747, 420, 746, 420, 745, 420, 744, 420, 743, 420, 742, 420, 741, 420, 740, 420, 739, 420, 738, 420, 737, 420, 736, 420, 735, 420, 734, 420, 733, 420, 732, 420, 731, 420, 730, 420, 729, 420, 728, 420, 727, 420, 726, 420, 725, 420, 724, 420, 723, 420, 722, 420, 721, 420, 720, 420, 719, 420, 718, 420, 717, 420, 716, 420, 715, 420, 714, 420, 713, 420, 712, 420, 711, 420, 710, 420, 709, 420, 708, 420, 707, 420, 706, 420, 705, 420, 704, 420, 703, 420, 702, 420, 701, 420, 700, 420, 699, 420, 698, 420, 697, 420, 696, 420, 695, 420, 694, 420, 693, 420, 692, 420, 691, 420, 690, 420, 689, 420, 688, 420, 687, 420, 686, 420, 685, 420, 684, 420, 683, 420, 682, 420, 681, 420, 680, 420, 679, 420, 678, 420, 677, 420, 676, 420, 675, 420, 674, 420]], [[802, 446, 801, 447, 800, 447, 799, 447, 798, 447, 797, 447, 796, 447, 795, 447, 794, 448, 793, 448, 792, 448, 792, 449, 792, 450, 793, 451, 794, 452, 795, 453, 796, 453, 797, 453, 798, 454, 799, 454, 800, 455, 801, 456, 801, 457, 802, 458, 802, 459, 802, 460, 803, 461, 803, 462, 803, 463, 803, 464, 804, 465, 804, 466, 804, 467, 805, 468, 805, 469, 805, 470, 805, 471, 806, 472, 806, 473, 806, 474, 807, 475, 807, 476, 807, 477, 808, 478, 808, 479, 808, 480, 808, 481, 808, 482, 808, 483, 808, 484, 808, 485, 808, 486, 808, 487, 809, 488, 810, 489, 811, 489, 812, 489, 813, 489, 814, 489, 815, 489, 816, 489, 817, 489, 818, 489, 819, 489, 820, 489, 821, 489, 822, 488, 823, 488, 824, 487, 824, 486, 825, 485, 825, 484, 825, 483, 825, 482, 825, 481, 826, 480, 826, 479, 826, 478, 826, 477, 827, 476, 827, 475, 827, 474, 828, 473, 828, 472, 828, 471, 828, 470, 827, 469, 827, 468, 827, 467, 827, 466, 826, 465, 826, 464, 826, 463, 825, 462, 825, 461, 824, 460, 823, 460, 822, 459, 821, 459, 820, 459, 819, 459, 818, 459, 817, 459, 816, 459, 815, 459, 814, 459, 813, 459, 812, 459, 811, 459, 810, 459, 809, 459, 808, 458, 808, 457, 808, 456, 809, 455, 809, 454, 809, 453, 810, 452, 811, 451, 812, 450, 812, 449, 813, 448, 813, 447, 812, 447, 811, 446, 810, 446, 809, 446, 808, 446, 807, 446, 806, 446, 805, 446, 804, 446, 803, 446]], [[1863, 468, 1862, 469, 1861, 469, 1860, 469, 1859, 469, 1858, 469, 1857, 470, 1856, 470, 1855, 471, 1855, 472, 1854, 473, 1853, 473, 1852, 474, 1853, 475, 1853, 476, 1853, 477, 1854, 478, 1854, 479, 1853, 480, 1853, 481, 1852, 482, 1852, 483, 1851, 484, 1850, 485, 1849, 486, 1848, 487, 1847, 488, 1846, 489, 1845, 490, 1844, 491, 1843, 492, 1843, 493, 1843, 494, 1842, 495, 1842, 496, 1842, 497, 1842, 498, 1842, 499, 1841, 500, 1841, 501, 1841, 502, 1841, 503, 1841, 504, 1840, 505, 1840, 506, 1840, 507, 1840, 508, 1840, 509, 1840, 510, 1840, 511, 1840, 512, 1840, 513, 1840, 514, 1840, 515, 1840, 516, 1840, 517, 1840, 518, 1840, 519, 1839, 520, 1839, 521, 1839, 522, 1839, 523, 1839, 524, 1839, 525, 1839, 526, 1839, 527, 1839, 528, 1839, 529, 1839, 530, 1839, 531, 1839, 532, 1839, 533, 1840, 534, 1840, 535, 1841, 536, 1841, 537, 1842, 538, 1842, 539, 1843, 540, 1843, 541, 1844, 542, 1844, 543, 1845, 544, 1846, 545, 1847, 545, 1848, 545, 1849, 546, 1850, 546, 1851, 545, 1852, 544, 1853, 543, 1854, 542, 1855, 541, 1855, 540, 1855, 539, 1855, 538, 1856, 537, 1856, 536, 1856, 535, 1856, 534, 1856, 533, 1856, 532, 1856, 531, 1857, 530, 1857, 529, 1857, 528, 1857, 527, 1858, 526, 1859, 525, 1859, 524, 1860, 523, 1860, 522, 1861, 521, 1862, 520, 1862, 519, 1862, 518, 1862, 517, 1862, 516, 1863, 515, 1863, 514, 1863, 513, 1863, 512, 1863, 511, 1863, 510, 1863, 509, 1863, 508, 1862, 507, 1862, 506, 1862, 505, 1862, 504, 1862, 503, 1861, 502, 1861, 501, 1862, 500, 1863, 500, 1864, 499, 1864, 498, 1865, 497, 1865, 496, 1865, 495, 1866, 494, 1866, 493, 1865, 492, 1864, 492, 1863, 491, 1862, 490, 1863, 489, 1863, 488, 1864, 487, 1864, 486, 1864, 485, 1865, 484, 1865, 483, 1866, 482, 1867, 481, 1868, 480, 1869, 479, 1870, 478, 1871, 478, 1872, 477, 1873, 477, 1874, 477, 1875, 476, 1876, 476, 1877, 475, 1876, 474, 1875, 474, 1874, 474, 1873, 474, 1872, 474, 1871, 474, 1870, 475, 1869, 475, 1868, 475, 1867, 474, 1867, 473, 1867, 472, 1867, 471, 1866, 470, 1866, 469, 1866, 468, 1865, 468, 1864, 468]]], ‘gt_classes’: array([1, 1, 2, 3, 3, 3, 3, 3, 3, 3, 3, 8, 8], dtype=int32), ‘seg_areas’: array([ 665。, 923。, 2270。, 2129。, 1636。, 6072。, 118。, 1293。,

5248。, 8739。, 17424。, 830。, 1479。], dtype=float32),

‘gt_overlaps’: <13x9 sparse matrix of type ‘

with 13 stored elements in Compressed Sparse Row format>,

‘is_crowd’: array([False, False, False, False, False, False, False, False, False,

False, False, False, False]),

‘box_to_gt_ind_map’: array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], dtype=int32),

‘max_classes’: array([1, 1, 2, 3, 3, 3, 3, 3, 3, 3, 3, 8, 8]),

‘max_overlaps’: array([1。, 1。, 1。, 1。, 1。, 1。, 1。, 1。, 1。, 1。, 1。, 1。, 1。], dtype=float32),

‘bbox_targets’: array([[1。, 0。, 0。, 0。, 0。],

[1。, 0。, 0。, 0。, 0。],

[2。, 0。, 0。, 0。, 0。],

[3。, 0。, 0。, 0。, 0。],

[3。, 0。, 0。, 0。, 0。],

[3。, 0。, 0。, 0。, 0。],

[3。, 0。, 0。, 0。, 0。],

[3。, 0。, 0。, 0。, 0。],

[3。, 0。, 0。, 0。, 0。],

[3。, 0。, 0。, 0。, 0。],

[3。, 0。, 0。, 0。, 0。],

[8。, 0。, 0。, 0。, 0。],

[8。, 0。, 0。, 0。, 0。]], dtype=float32)},

{},

。。。

重要欄位解釋:

boxes:該張圖片中所有例項的bbox

segms:該張圖片中每個例項的mask

bbox_targets: 作為計算邊框迴歸和分類損失時的label,第一位數字是該box下物體的類別,後四位數字是邊框位置微調的變換引數。因為網路不是直接對邊框的座標進行迴歸,而是迴歸anchor微調至gt的變換引數(具體如何變換,後續文章會講到)。由於本身就是gt,所以變換引數都為0。

2。2 __

getitem

__ 對每個roidb執行的操作

(1)首先,為影象隨機選擇一個預設的尺寸,縮放影象,翻轉影象(視roidb中flip引數而定),歸一化影象,將圖片格式從HWC轉換成CHW;

(2)其次,製作RPN的標籤, 核心函式

add_rpn_blobs

① 在不同的特徵圖上生成anchors。不同的特徵圖對應不同的anchor_base_size。

生成的all anchor如下表所示。儘管只使用了一種scale,但配合FPN使用了不同大小的anchor_base_size

同樣得到了不同大小的anchor,以檢測不同大小的物體。

\begin{array}{c|c} \text{rpn_feat_stride} & \text{feature maps} &\text{anchor base_size}&\text{anchor scaled size}&\text{ratios}&\text{nums}\\ \hline 4 & p2 ,H/4 * W/4&4*4&32*32&(0.5, 1, 2)&(max(H/4, W/4))^2*3 \\ 8 & p3 ,H/8 * W/8&8*8&64*64&(0.5, 1, 2)&(max(H/8, W/8))^2*3 \\ 16 & p4 ,H/16 * W/16&16*16&128*128&(0.5, 1, 2)&(max(H/16, W/16))^2*3 \\ 32& p5,H/32 * W/32&32*32&256*256&(0.5, 1, 2)&(max(H/32, W/32))^2*3 \\ 64& p6 ,H/64 * W/64&64*64&512*512&(0.5, 1, 2)&(max(H/64, W/64))^2 *3\\ \end{array}\\

foa即Field of Anchor, 資料格式及含義如下:

{

“field of anchors”: 二維陣列,圖片上anchor的座標,

“number_cell_anchors”: 3

“stride”: 即rpn_feat_stride, 同樣是特徵圖相比於原圖的縮小倍數,也即特徵圖上移動一個pixel對應原圖上移動的pixel數

“octave”: None

“aspect”: None

}

② 確定正負樣本

計算每一個anchor和gt box的重疊面積。

對於每一個gt box, 都有與它重疊面積最大的anchor,將其label設定為1;

對於與gt box 重疊面積超過閾值(0。7)的,將其label設定為1;

根據設定的正負樣本總和(N)和正樣本比例係數(

\alpha

)這兩個超引數進行正負樣本的篩選;

正樣本篩選:正樣本最多個數為總和乘以正樣本比例(

N*\alpha

),若當前正樣本個數(N1)超過規定的最多個數,則進行隨機篩選。

負樣本篩選:從諸多負樣本中篩選出

N-min(N_1, N*a)

個, 將label設定為0。其餘非正負樣本的label設定為-1

③ 計算正樣本的邊框變換系數和inside,outside權重

bbox_inside_weights指用於計算邊框迴歸損失時的權重,由於只計算正樣本的損失,所以只將正樣本的bbox_inside_weights設定為[1, 1, 1,1],其餘設定成[0,0,0,0]

bbox_outside_weights是為了平衡分類損失和迴歸損失而設定的引數,其大小為正負樣本個數分之一,即

1/N

(3)最後製作分割的標籤

分割的標籤包含多個部分,一一列舉如下。

① seg gt:語義分割的gt影象,每個畫素的值為對應的trainId

② gt classes: 每一個box的類別

③ mask gt :每一個box內物體的mask, 值為0, 1

④ seg roi gt: seg gt 在box內的部分

3。 資料輸出

整個Cityscapes類最終輸出包含data、label、index

data:

{

“data”: ndarray, shape(3, H, W), # 預處理後的影象

“img_info”: [[H, W, scale_factor]]

}

label:

{‘roidb’:

{‘boxes’: array, shape=(17,4)

‘segms’: list, len=17, list裡每個元素是object的polygon標註,同一個object可能包含多個polygon

‘seg_areas’: ndarray 面積

‘gt_classes’: array([1, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], dtype=int32),

‘gt_overlaps’:

‘is_crowd’:

‘box_to_gt_ind_map’: }

‘rpn_labels_fpn4’: ndarray, shape=(1, 3, 512, 512) 每一個anchor的前後景標註

‘rpn_bbox_targets_fpn4’: ndrray, shape=(1, 12, 512, 512), 每一個anchor邊框的迴歸目標,非前景為0

‘rpn_bbox_inside_weights_fpn4’: ndrray, shape=(1, 12, 512, 512), 每一個anchor邊框的迴歸目標,非前景為[0, 0, 0, 0], 前景為[1, 1, 1,1]

‘rpn_bbox_outside_weights_fpn4’: ndrray, shape=(1, 12, 512, 512), 每一個anchor邊框的迴歸目標,非前景為[0, 0, 0, 0], 前景為[1, 1, 1,1]/正負樣本總和

‘rpn_labels_fpn8’: 與fpn4相似,寬高變成256

‘rpn_bbox_targets_fpn8’:

‘rpn_bbox_inside_weights_fpn8’:

‘rpn_bbox_outside_weights_fpn8’:

‘rpn_labels_fpn16’:128

‘rpn_bbox_targets_fpn16’:

‘rpn_bbox_inside_weights_fpn16’:

‘rpn_bbox_outside_weights_fpn16’:

‘rpn_labels_fpn32’:64

‘rpn_bbox_targets_fpn32’:

‘rpn_bbox_inside_weights_fpn32’:

‘rpn_bbox_outside_weights_fpn32’:

‘rpn_labels_fpn64’:32

‘rpn_bbox_targets_fpn64’:

‘rpn_bbox_inside_weights_fpn64’:

‘rpn_bbox_outside_weights_fpn64’:

‘seg_gt’: ndarray shape(H, W)

‘gt_classes’: array([1, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], dtype=int32),

‘mask_gt’: ndarray, shape(17, H, W),object的語義標註, 值為0,1}