上海住房城乡建设部网站,软装工作室,上海公关公司排名,湖南郴州市有几个县参考#xff1a;http://nil.csail.mit.edu/6.824/2021/labs/lab-mr.html task1: 熟悉讲义#xff0c;尤其是搞明白如何运行测试程序(完成) ------------------------------------------------ start
先看 Introduction 我们的目标#xff1a;构建一个MapReduce系统。 细节http://nil.csail.mit.edu/6.824/2021/labs/lab-mr.html task1: 熟悉讲义尤其是搞明白如何运行测试程序(完成) ------------------------------------------------ start
先看 Introduction 我们的目标构建一个MapReduce系统。 细节实现一个工作进程调用应用程序的Map和Reduce函数并处理文件的读写同时还将实现一个协调进程负责分配任务给工作进程并应对失败的工作进程。构建的内容类似于MapReduce论文中描述的内容。注意实验中使用“协调者”而不是论文中的“主节点”。
细节在后边 先根据讲义下载 LAB 代码
git clone git://g.csail.mit.edu/6.824-golabs-2021 6.824看讲义代码中已经有了 word-count 和 text indexer 两个应用。同时还有一个极简单的 MapReduce 实现在一个进程中一次只运行一个 map 和 一个reduce。 我们电脑上没 go 先安装安装步骤如下 (WSL2 ubuntu20.04)
wget -qO- https://golang.org/dl/go1.15.8.linux-amd64.tar.gz | sudo tar xz -C /usr/local
sudo ln -s /usr/local/go/bin/go /usr/local/bin/go安装完毕我们试着按照讲义运行 word count 这个应用
cd ~/6.824
cd src/main
go build -race -buildmodeplugin ../mrapps/wc.go
go run -race mrsequential.go wc.so pg*.txt
more mr-out-0出现了单词统计结果 看来顺利运行了
看看后边的内容 这一段概述了要做的事情作业目标
下面内容介绍了一些代码大致位置 1.main函数包括对 协调者 和 工作者 的调动在 main/mrcoordinator.go 里不应修改 2.我们的实现应该放在 mr/coordinator.go, mr/worker.go 和 mr/rpc.go 里 更多内容看后边 仔细阅读了后边的内容总结了运行测试程序的方式 1.修改 mr/coordinator.go Done 函数中的 返回值把 return false 改成 return true 2.在 main 文件夹下运行 bash test-mr.sh 看到输出如下
*** Starting wc test.
2024/09/29 10:39:19 rpc.Register: method Done has 1 input parameters; needs exactly three
sort: cannot read: mr-out*: No such file or directory
cmp: EOF on mr-wc-all which is empty
--- wc output is not the same as mr-correct-wc.txt
--- wc test: FAIL
*** Starting indexer test.
2024/09/29 10:39:21 rpc.Register: method Done has 1 input parameters; needs exactly three
sort: cannot read: mr-out*: No such file or directory
cmp: EOF on mr-indexer-all which is empty
--- indexer output is not the same as mr-correct-indexer.txt
--- indexer test: FAIL
*** Starting map parallelism test.
2024/09/29 10:39:22 rpc.Register: method Done has 1 input parameters; needs exactly three
cat: mr-out*: No such file or directory
--- saw 0 workers rather than 2
--- map parallelism test: FAIL
cat: mr-out*: No such file or directory
--- map workers did not run in parallel
--- map parallelism test: FAIL
*** Starting job count test.
2024/09/29 10:39:24 rpc.Register: method Done has 1 input parameters; needs exactly three
cat: mr-out*: No such file or directory
test-mr.sh: line 170: [: : integer expression expected
--- job count test: PASS
*** Starting early exit test.
2024/09/29 10:39:25 rpc.Register: method Done has 1 input parameters; needs exactly three
sort: cannot read: mr-out*: No such file or directory
sort: cannot read: mr-out*: No such file or directory
--- early exit test: PASS
*** Starting crash test.
2024/09/29 10:39:26 rpc.Register: method Done has 1 input parameters; needs exactly three
sort: cannot read: mr-out*: No such file or directory
cmp: EOF on mr-crash-all which is empty
--- crash output is not the same as mr-correct-crash.txt
--- crash test: FAIL
*** FAILED SOME TESTS本 task 完成
task1: 熟悉讲义尤其是搞明白如何运行测试程序(完成) ------------------------------------------------ end
task2: 该明白测试程序的原理尤其是如何测试代码是并行运行的 ------------------------------------------------ start
TODOhere
task2: 该明白测试程序的原理尤其是如何测试代码是并行运行的 ------------------------------------------------ end
TODO: 看看 word count 和 MapReduce 的联动
TODO: here
